Data operation monitoring method, system and device and storage medium

Abstract

The invention discloses a data operation monitoring method, system and device and a storage medium. The method comprises the following steps: adding a log recording section in a target program through section annotation; wherein the target program is used for executing at least one data operation task; if the target program is called, recording calling information corresponding to the target program through the log recording aspect; wherein the calling information comprises calling time, calling parameters and a calling executor; if the target program is executed, recording operation result information corresponding to the target program through the log recording aspect; and monitoring data operation based on the operation result information and the calling information. According to the method, recording of the calling information or the operation result information is achieved through the section annotation technology, the depth and details of data recording operation are improved, and then the processing efficiency of system problems is improved. The method can be widely applied to the technical field of computers.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data operation monitoring method, system, device, and storage medium. Background Technology

[0002] In modern distributed systems, data transmission and manipulation are highly complex, involving multiple services, interfaces, and data structures. To ensure transparency, compliance, and security in business operations, it is essential to meticulously record the data flow of each operation. Current logging methods often lack real-time monitoring of the entire process of method execution and data transfer, typically only recording the results of operations without tracing every detail of the data manipulation. This makes it difficult to trace and troubleshoot anomalies when problems arise. Summary of the Invention

[0003] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0004] Therefore, the purpose of this invention is to provide a method, system, device, and storage medium for in-depth data operation monitoring.

[0005] To achieve the aforementioned technical objectives, one aspect of this invention provides a data operation monitoring method, comprising the following steps: adding a logging aspect to a target program using aspect annotations; wherein the target program is used to perform at least one data operation task; if the target program is called, recording the corresponding call information of the target program through the logging aspect; wherein the call information includes call time, call parameters, and caller; if the target program is executed, recording the corresponding operation result information of the target program through the logging aspect; wherein the operation result information includes execution result or exception information; and monitoring the data operation based on the operation result information and the call information. This embodiment of the application uses aspect annotation technology to record call information or operation result information, which is beneficial for improving the depth and detail of recording data operations, thereby improving the efficiency of system problem handling.

[0006] In some embodiments, the data operation monitoring method further includes:

[0007] Add annotations to the target program to trigger a data validation aspect before the target program is executed, so as to validate the parameters of the target program.

[0008] If invalid data is found through the data verification aspect, it is recorded and an error message is returned, and the target program is not executed.

[0009] In some embodiments, the data operation monitoring method further includes:

[0010] obtaining a to-be-checked parameter of the target program through the around notification connection point;

[0011] determining a checking strategy according to a data type of the to-be-checked parameter;

[0012] if the to-be-checked parameter is a nested structure, performing recursive checking or data checking on the to-be-checked parameter through a data checking framework;

[0013] if it is found through the data checking that data is abnormal, recording data abnormality information.

[0014] In some embodiments, the data operation monitoring method further comprises:

[0015] adding an exception throwing after notification annotation in the target program, so that when the target program throws an exception, the exception information is captured;

[0016] extracting type, location and stack information in the exception information, and according to a preset exception handling strategy, storing the exception information according to levels in a log system, recording the exception information or providing recovery measures.

[0017] In some embodiments, the log recording aspect comprises a pre-notification, an around notification and a return after notification.

[0018] The recording, by the log recording aspect, of the calling information corresponding to the target program comprises:

[0019] through the pre-notification, recording calling parameters before the target program is executed;

[0020] through the around notification, recording time stamps before and after the target program is executed, and determining calling time;

[0021] through the around notification, obtaining a calling executor according to preset parameters;

[0022] converting the calling information into structured log data according to a preset data format;

[0023] The recording, by the log recording aspect, of the operation result information corresponding to the target program comprises:

[0024] through the return after notification, recording operation result information of the target program;

[0025] converting the operation result information into structured log data according to a preset data format;

[0026] calling a log framework to output the structured log data.

[0027] In some embodiments, the data operation monitoring method further comprises:

[0028] storing the calling information and the operation result information in a database according to a preset data format, comprising:

[0029] determining elements of a database format template according to data characteristics of the calling information and the operation result information; the elements comprising data, time, and operation related information;

[0030] constructing a database structure conforming to the preset data format according to the elements, so as to store the calling information and the operation result information in the database according to the preset data format.

[0031] In some embodiments, if the to-be-verified parameter does not belong to a nested structure, data verification is performed through the following steps:

[0032] obtaining a parameter type of the to-be-verified parameter, and determining whether the parameter type is consistent with a preset parameter type;

[0033] if the to-be-verified parameter is a numerical value type parameter, determining whether a numerical value of the to-be-verified parameter is in a preset numerical value interval;

[0034] if the to-be-verified parameter contains a string or a date, performing format verification on the string or the date.

[0035] In another aspect, an embodiment of the present application provides a data operation monitoring system, comprising:

[0036] a first module configured to add a log recording aspect in a target program through an aspect annotation; wherein the target program is configured to perform a task of at least one data operation;

[0037] a second module configured to record calling information corresponding to the target program through the log recording aspect if the target program is called; wherein the calling information comprises calling time, calling parameters, and calling executors;

[0038] a third module configured to record operation result information corresponding to the target program through the log recording aspect if the target program is executed; wherein the operation result information comprises execution results or exception information;

[0039] a fourth module configured to monitor data operation based on the operation result information and the calling information.

[0040] In another aspect, an embodiment of the present application provides a data operation monitoring device, comprising:

[0041] at least one processor;

[0042] at least one memory configured to store at least one program;

[0043] The at least one program, when executed by the at least one processor, causes the at least one processor to implement the data operation monitoring method.

[0044] In another aspect, an embodiment of the present application provides a storage medium having stored therein a processor-executable program for implementing the data operation monitoring method when executed by a processor.

[0045] The present application has at least the following beneficial effects: The method provided by the present application includes: adding a log recording aspect in a target program through aspect annotation, wherein the target program is used to perform a task of at least one data operation; if the target program is called, recording calling information corresponding to the target program through the log recording aspect, wherein the calling information includes calling time, calling parameters, and calling performer; if the target program is executed, recording operation result information corresponding to the target program through the log recording aspect, wherein the operation result information includes execution result or exception information; and monitoring the data operation based on the operation result information and the calling information. The present application records the calling information or the operation result information through aspect annotation technology, which is beneficial to improving the depth and details of recording the data operation, and further improving the processing efficiency of system problems. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly describing some embodiments of the technical solutions in the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0047] Figure 1 A flowchart of an embodiment of the data operation monitoring method provided by the present application;

[0048] Figure 2 A flowchart of another embodiment of the data operation monitoring method provided by the present application;

[0049] Figure 3 A flowchart of an embodiment of the log management process provided by the present application;

[0050] Figure 4 A flowchart of an embodiment of the data verification process provided by the present application;

[0051] Figure 5A structural schematic diagram of an embodiment of the data operation monitoring system provided by the present application;

[0052] Figure 6 A structural schematic diagram of an embodiment of the data operation monitoring device provided by the present application. DETAILED DESCRIPTION

[0053] Embodiments of the present application are described in detail below with reference to the accompanying drawings, in which the same or similar elements or elements having the same or similar functions are denoted by the same or similar reference numerals throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and cannot be understood as a limitation of the present application. For the step numbers in the following embodiments, they are only set for the convenience of explanation and description, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0054] In modern distributed systems, the transmission and operation process of data is very complex, involving multiple services, interfaces and data structures. In order to ensure the transparency, compliance and security of business operations, it is necessary to record the data flow process of each operation in detail. However, the traditional log recording method usually lacks real-time monitoring of the whole process of method execution and data transmission, often only records the results of the operation, and cannot track every detail of the operation. This makes it difficult to trace and troubleshoot exceptions when problems occur. Specifically, the following problems exist:

[0055] 1. Lack of depth in log recording: traditional log recording can only simply record the results of method execution, lacking detailed operation steps, incoming parameters, return results and other key information, making it difficult to meet the needs of business safety and compliance.

[0056] 2. Lack of real-time parameter checking: traditional log recording methods usually do not perform type verification, range checking and data consistency checking on incoming parameters, which may cause illegal parameters to enter the business logic layer, causing data errors or security vulnerabilities.

[0057] 3. Difficulty in operation tracing: operation records in traditional systems are often scattered in different modules, lacking unified and clear operation flow, making it difficult for developers to quickly locate the source of the problem when problems occur.

[0058] 4. Lack of flexibility and scalability: traditional log mechanisms cannot flexibly adapt to different data types and complex data structures, and cannot effectively handle diverse data verification needs.

[0059] To solve this problem, the aspect annotation (Aspect-Oriented Programming, AOP) technology provides a method of dynamically intercepting and enhancing program behavior during program execution, which can automatically perform log recording, parameter verification, exception handling and other operations before and after method calling without changing the original code logic, thereby providing traceable operation flow information for each operation. This technology not only improves the security of the system, but also provides detailed information to help developers quickly diagnose and repair when problems occur.

[0060] The present application aims to build an efficient, transparent and extensible data operation monitoring system by introducing aspect annotation technology, which can comprehensively record the process of each method call, including input parameters, method execution time, return results, exception information, etc., while performing real-time parameter verification and data compliance monitoring to ensure that each data transmission meets the security and compliance requirements.

[0061] The data operation monitoring method and system according to the embodiments of the present application will be described in detail below with reference to the accompanying drawings. First, the data operation monitoring method according to the embodiments of the present application will be described with reference to the accompanying drawings.

[0062] Referring to Figure 1 In the embodiments of the present application, a data operation monitoring method is provided. The data operation monitoring method in the embodiments of the present application can be applied in a terminal, can be applied in a server, and can also be software running in a terminal or a server, etc. The terminal can be a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto. The server can be a standalone physical server, can be a server cluster or a distributed system composed of multiple physical servers, can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The data operation monitoring method in the embodiments of the present application mainly includes the following steps:

[0063] S100: adding a log recording aspect in a target program through aspect annotation; wherein the target program is used to perform at least one data operation task;

[0064] S200: if the target program is called, recording the calling information corresponding to the target program through the log recording aspect; wherein the calling information includes calling time, calling parameters, and calling performer;

[0065] S300: if the target program is executed, recording the operation result information corresponding to the target program through the log recording aspect; wherein the operation result information includes execution result or exception information;

[0066] S400: monitoring the data operation based on the operation result information and the calling information.

[0067] It can be understood that the data operation monitoring method provided by the application can be used in any occasion requiring data monitoring. For example, for the medical and financial industries, relevant enterprises can monitor and record data operation by using the method provided by the application, which facilitates subsequent data auditing and other work. On the other hand, enterprises in any industry can also monitor and record data operation by using the method provided by the application, which facilitates quick troubleshooting and checking of employee behavior when the business system fails. Of course, the application does not limit the specific application scenarios. For a business system requiring data monitoring, a log recording aspect is added in the method corresponding to the target program (business program) through aspect annotation. If the target program is called, the calling information is recorded through the log recording aspect, and the target program is executed to record the execution result or exception information, thereby obtaining operation flow information; wherein the calling information includes calling time, calling parameters and calling executor, and the operation flow information includes calling information, execution result and exception information; the data operation is monitored based on the operation flow information.

[0068] It can be understood that the data operation monitoring method provided by the application can be used in any scenario requiring log information. For example, for a financial APP, if the target user needs to be risk-controlled or the target user initiates a fund interaction request, the operation information of the target user on the data needs to be recorded in detail, and the data operation monitoring method provided by the application can be used to monitor the data operation. Specifically, a first request of a target user is received, the first request is a request related to fund interaction, including a loan request, a payment request, etc.; a log recording aspect is added in the program related to the target user, the calling information and the operation result information of the program are recorded through the log recording aspect, and the data operation is monitored according to the calling information and the operation result information. In some other scenarios, on the operation and maintenance side of the back end, if it is in a period of large business demand, a relevant holiday period or a period defined according to business demand, efficient operation of the business system is required, and the detailed calling information and operation result information can be recorded by using the data operation monitoring method provided by the application, thereby facilitating quick troubleshooting when the system fails.

[0069] It should be noted that the target program is a carrier of data operation, and the data operation record is the data flow conversion process corresponding to each operation of the target program. The calling information and the operation result information in the present application can be used as operation pipeline information to record the process of data operation, including the calling information of the target program and the execution result of the target program. Through the operation pipeline information, the abnormal information of the program can be intuitively understood, and the monitoring of the data operation can be realized. Through the aspect annotation technology, the present application adds a log record aspect, so that the related operation pipeline information can be recorded during the execution of the target program, and the operation pipeline information can be stored in a preset data format, thereby facilitating the subsequent query of the operation pipeline information.

[0070] In some embodiments, the data operation monitoring method provided by the present application further comprises:

[0071] An annotation is added to the target program, so that the data verification aspect is triggered before the target program is executed, and the parameters of the target program are verified.

[0072] If illegal data is found through the data verification aspect, error information is recorded and returned, and the target program is not executed.

[0073] In some embodiments, the data operation monitoring method provided by the present application further comprises:

[0074] The to-be-verified parameter of the target program is obtained through the around notification connection point;

[0075] According to the data type of the to-be-verified parameter, a verification strategy is determined;

[0076] If the to-be-verified parameter is a nested structure, recursive checking or data verification of the to-be-verified parameter is performed through a data verification framework;

[0077] If data exception is found through data verification, data exception information is recorded.

[0078] In the present application, according to the requirement, a log record aspect and a data verification aspect can be established respectively, or any aspect with other functions can be realized. A single aspect can also be established to realize log recording and data verification. The present application does not limit the classification of the established aspect in detail. The parameters of the target program in the present application can be calling parameters or other types of parameters. When the data / parameters are verified, the corresponding verification strategy can be selected according to the type of the parameter, the verification strategy is designed into the data verification aspect, and the verification accuracy is improved. For the to-be-verified parameter with a complex data structure, the data verification can be performed with the help of a data verification framework.

[0079] The data verification in the present application includes but is not limited to data type verification, data range verification, data format verification, data legality verification, etc. It can be understood that if the exception found by the data verification is an exception that does not affect the safety of the system, the data exception can be recorded only through the aspect, and the execution progress of the target program is not interrupted. If the exception found by the data verification is an exception that affects the safety of the system, such as finding illegal data, an exception can be thrown to terminate the subsequent execution of the target program.

[0080] In some embodiments, the data operation monitoring method provided by the present application further comprises:

[0081] Adding an exception throwing notification annotation in the target program, so that the target program captures exception information when throwing an exception;

[0082] Extracting the type, location and stack information in the exception information, and storing the exception information according to a preset exception handling strategy according to the level, recording the exception information or providing recovery measures.

[0083] The present application can also provide an exception capturing aspect for recording exception information in the execution process of the target program. The exception information can be any exception information. The present application can perform different recording processing for different exception information, thereby improving the efficiency of subsequent problem processing.

[0084] In some embodiments, the log recording aspect includes pre-notification, around notification, and post-return notification.

[0085] The call information corresponding to the target program recorded by the log recording aspect includes:

[0086] Through pre-notification, record the call parameters before the target program is executed;

[0087] Through the around notification, record the time stamps before and after the execution of the target program, and determine the call time;

[0088] Through the around notification, the call executor is obtained according to the preset parameters;

[0089] The call information is converted into structured log data according to a preset data format;

[0090] The operation result information corresponding to the target program recorded by the log recording aspect includes:

[0091] Through the post-return notification, record the operation result information of the target program;

[0092] The operation result information is converted into structured log data according to a preset data format;

[0093] invoking a log framework to output the structured log data.

[0094] The application receives preset parameters (exemplarily, ProceedingJoinPoint parameters) in the method of the surrounding notification; and obtains the calling executor information of the program through the ProceedingJoinPoint parameters, including the class name and the method name of the calling executor.

[0095] In some embodiments, the data operation monitoring method provided by the application further comprises:

[0096] storing the calling information and the operation result information in the database according to the preset data format, including:

[0097] determining the elements of the database format template according to the data characteristics of the calling information and the operation result information; the elements include data, time, and operation related information;

[0098] constructing a database structure conforming to the preset data format according to the elements, so as to store the calling information and the operation result information in the database according to the preset data format.

[0099] In some embodiments, if the to-be-verified parameter does not belong to a nested structure, data verification is performed through the following steps:

[0100] obtaining the parameter type of the to-be-verified parameter, and determining whether the parameter type is consistent with a preset parameter type;

[0101] if the to-be-verified parameter is a numerical value type parameter, determining whether the numerical value of the to-be-verified parameter is in a preset numerical value interval;

[0102] if the to-be-verified parameter contains a string or a date, performing format verification on the string or the date.

[0103] If the to-be-verified parameter does not belong to a nested structure, data verification can be performed through conventional data verification logic. Specifically, it is determined whether the parameter type is consistent with a preset parameter type, and if not, data abnormal information of the inconsistency is recorded. If the to-be-verified parameter is a numerical value type parameter, it is determined whether the numerical value of the to-be-verified parameter is in a preset numerical value interval, and if not, data abnormal information of the numerical value abnormality is recorded. If the to-be-verified parameter contains a string or a date, format verification is performed on the string or the date, and if the format verification fails, data abnormal information of the format abnormality is recorded. The preset parameter type is a parameter type acceptable for current input, and the preset numerical value interval is related to specific business requirements and can be set according to actual conditions, which is not limited by the application.

[0104] In some embodiments, the data operation monitoring method provided by the application further comprises:

[0105] The data operation monitoring architecture comprises:

[0106] A log recording unit is configured to record the calling information and operation result information before and after the execution of the target program.

[0107] A data verification unit is configured to verify the data before the execution of the target program.

[0108] An exception capturing unit is configured to record the exception thrown during the execution of the target program.

[0109] The data operation monitoring method provided by the application will be described in detail in the following specific embodiment:

[0110] The core target of the application includes the following aspects:

[0111] 1. Comprehensive recording of data operations: the process of each method call is recorded in detail, including input parameters, execution time, return results, exception information, etc., to ensure the traceability of the operation.

[0112] 2. Real-time parameter verification: before and after each method execution, the incoming parameters are verified for type and data range, to ensure the legality and compliance of the data.

[0113] 3. Efficient log management: on the premise of not affecting system performance, efficient and accurate operation flow recording is achieved, which is convenient for later analysis and problem tracing.

[0114] 4. Flexible data structure adaptation: aspect annotation can flexibly adjust the verification strategy according to different data types and structures, ensuring the adaptability of the system to complex data scenarios.

[0115] It can be understood that aspect programming realizes the enhancement of the program execution process through aspects (Aspect), which is usually used for functions such as log recording, permission control, performance monitoring, etc. By configuring aspect methods through annotation, additional operations can be performed before (pre-enhancement) or after (post-enhancement) some operations. It has the following advantages:

[0116] 1. Decoupling and code reuse.

[0117] Pre- and post-insertion operations (such as data verification, log recording, etc.) are often repeated in multiple places. Using aspect annotation can separate these functions from business logic, avoiding repeated code.

[0118] For example, data verification, monitoring logs, etc. can be handled uniformly through aspects, reducing redundant code in each method.

[0119] 2. Enhance the maintainability of the code.

[0120] Aspect programming makes the code irrelevant to the core business logic can be managed centrally. When the need to modify or optimize the log, permission verification, etc. Function, just modify the logic in the aspect, without modifying each specific business method.

[0121] 3. Improve the readability of the code.

[0122] The operation of inserting before and after the abstraction to the aspect, can keep the simplicity of business code, so that the developer can focus on the core logic.

[0123] For example, the logic of data verification, transaction management and other operations are separated from the business processing logic, and when reading the code, you can focus on the implementation of business functions.

[0124] 4. Enhance the flexibility of the code.

[0125] Aspect annotation allows you to dynamically decide whether to apply some enhancement logic at runtime. For example, you can choose whether to enable a certain aspect at runtime according to different needs, which increases the flexibility of the system.

[0126] 5. Improve the scalability of the system.

[0127] Since the aspect logic is separated into a separate module, they can be extended independently. For example, monitoring, data validation and other functions can be independently extended as the demand changes, without modifying the existing business code.

[0128] For example, there is a user registration method, if implemented before and after the insertion of some monitoring and data validation operations, can be done in the following way:

[0129] Pre-enhancement: Before the registration method is executed, the input data is verified to ensure that the user provides the data meets the requirements.

[0130] Post-enhancement: After the registration method is executed, log or performance monitoring is recorded.

[0131] Aspect annotation (AOP) itself is used to provide separation of cross-cutting concerns (such as logging, performance monitoring, transaction management, permission control, etc.). Its core purpose is to abstract these cross-cutting concerns from business logic, achieving better decoupling and higher code reuse. Referring to Figure 2 The present application achieves:

[0132] Logging: AOP aspect can be used to record logs before and after the method (i.e. the target program in the present application) is called.

[0133] Parameter verification: Through AOP aspect, the parameters can be verified before the method is executed.

[0134] Exception capture: AOP aspects can capture exceptions during method execution and perform appropriate processing or logging.

[0135] For parameter checking and verification, it is common to obtain method parameters in aspects for checking, but sometimes it may involve complex parameter structures (such as objects, collections, etc.) and support for annotations.

[0136] For this, you can use the @Before (pre-notification / enhancement) or @Around (surrounding notification) enhancement to obtain the parameters of the method, and use JoinPoint or ProceedingJoinPoint (surrounding notification connection point) to obtain the parameters.

[0137] If the parameters are complex objects or contain nested objects, you may need to use reflection or tools provided by the framework to complete the verification.

[0138] For logging, logging is performed when the business method is executed, especially when there are many method parameters, and these information needs to be displayed reasonably in the log.

[0139] This application uses ProceedingJoinPoint in the aspect to obtain the parameters before method execution and the return value after execution, and records detailed logs. And use the log framework (such as SLF4J, Log4j) for log output.

[0140] For exception capture, capture exceptions during method execution and perform appropriate processing or logging. Exception capture may involve different types of exceptions, and different processing needs to be performed according to different exception types.

[0141] This application uses the @AfterThrowing (exception throwing notification) aspect annotation to capture exceptions during method execution. And according to the exception type, perform corresponding processing (such as logging, sending notifications, etc.).

[0142] In addition, parameter acquisition and verification: when the method parameters are complex, you may need to handle deep object verification, at which time you can use some mature verification frameworks to simplify the work.

[0143] Completeness of log information: when logging, make sure that the incoming parameters, return results, exceptions, etc. are complete, and avoid performance problems caused by too much log information.

[0144] Classification processing of exceptions: when capturing exceptions, different processing logic needs to be performed according to the exception type, such as different exceptions may require different log recording levels, error information or recovery measures.

[0145] On the other hand, about the architecture design of this application.

[0146] The present application inserts operation monitoring and data verification logic before and after method execution by introducing aspect annotation technology, so as to monitor and record the data operation process in real time. The aspect annotation integrates functions such as log recording, parameter verification, and exception capture, forming a comprehensive monitoring framework. The main components of the scheme include:

[0147] (1) Aspect annotation (AOP): Use aspect annotation technology to intercept before and after method execution, realize the enhancement of the method, record input and output parameters, execution time, return result, and exception information, etc.

[0148] (2) Log recording module (log recording unit): Before and after the method execution, record the parameters, execution time, return result and the occurred exception of each call, and generate complete operation flow information.

[0149] (3) Parameter verification module (data verification unit): Before method call, automatically perform parameter type checking, data format verification, data range checking, etc., to ensure that the input parameters meet the requirements of business logic. If the verification fails, the aspect annotation will automatically record and return error information.

[0150] (4) Exception handling module (exception capture unit): Capture and record the exceptions thrown in the method execution process, and record the exception information in detail, including exception type, stack information, etc., so as to facilitate the subsequent analysis of the developer.

[0151] Referring to Figure 3 The operation flow recording process of the present application can be that the aspect annotation automatically records logs before and after the target program call when each business operation is executed, and stores the operation information into the database or log management system according to the log requirements. The specific design is as follows:

[0152] (1) Input parameter recording: Before the method execution, record the parameter type and value through the aspect, to ensure that each operation can be traced.

[0153] (2) Method execution time recording: The aspect annotation records the execution time of each method, including the start time and the end time, which is used for performance analysis and optimization. The execution time can be determined by recording the time stamps before and after the method execution through the surrounding notification, and the log recorder records in detail.

[0154] (3) Return result recording: After the method execution, the aspect annotation records the return result type and value, to ensure the integrity of the operation.

[0155] (4) Exception Record: During method execution, if any exception occurs, the aspect annotation will capture and record detailed exception information, including exception type, location, stack information, etc.

[0156] (5) Operation Stream Storage: All operation records are stored in the operation stream table in the database, ensuring that each operation can be traced and analyzed later.

[0157] The elements of the database format template can be set according to actual needs. In some embodiments, it can include: auto-increment primary key, business name, operation type (and by defining a preset symbol, mark different operations, for example, 0-unknown, 1-insert, 2-update), operation method, page name, pre-operation DB data, request data, response data, user name, link name, operator, creation time, modification time, version number, extension information, etc. And the database can be defined as an operation stream record table, which is convenient for subsequent viewing and modification.

[0158] Referring to Figure 4 The parameter verification and exception handling process provided by the present application can be:

[0159] (1) Real-time verification of method parameters. Through aspect annotation before method invocation, the following verification is automatically performed on input parameters:

[0160] Type verification: Ensure that the type of the input parameter meets the expectations, such as string type cannot be passed in number type.

[0161] Range verification: According to business requirements, verify whether the numerical value type parameter is within a reasonable range, such as amount, date, etc.

[0162] Format verification: For special types such as strings or dates, perform format verification, such as date format yyyy-MM-dd, email format meets the standard.

[0163] (2) If illegal data is found during verification, the aspect annotation will immediately record and return error information to prevent illegal data from entering the business processing layer.

[0164] Regarding exception capture and processing, during method execution, the aspect annotation can capture all unhandled exceptions and record them in detail. Exception information includes:

[0165] (1) Exception type: such as NullPointerException, IllegalArgumentException, etc.

[0166] (2) Stack information: records the stack information when the exception occurs, helping developers quickly locate the problem.

[0167] Exception message: Record specific information of exceptions for problem troubleshooting and subsequent analysis.

[0168] The present application realizes transparent operation monitoring: through aspect annotation technology, each detail in the method calling process is recorded to ensure the traceability of operation process. Aspect programming (AOP) can automatically record the time of method execution, executor, incoming parameters, execution result and other information without modifying the original business logic, so as to conduct real-time monitoring and auditing. Such technology can not only provide detailed operation log, but also provide data support for later problem troubleshooting and performance optimization. The present application realizes real-time data verification: dynamically verifying the type, format and range of input parameters to ensure the compliance and security of data transmission. By strictly checking each parameter at the interface layer or service layer, it can effectively avoid illegal or non-compliant data flowing into the system, thereby reducing the risk of system failure or data leakage. Data verification can be divided into static verification (before data input) and dynamic verification (during data transmission) to ensure comprehensive security protection. The present application realizes efficient log management: each operation log is recorded through database and reasonable log management mechanism is designed to ensure efficient storage and fast query. Log recording includes but is not limited to user operation, system exception, data change and other key information. In order to ensure the efficient operation of the system, log files can be stored by time period or event type, and batch export and cleaning of logs are supported. Using the log system, the operation and maintenance team can quickly locate problems when an exception occurs, thereby minimizing the impact of system failure.

[0169] The present application introduces aspect annotation technology, real-time data verification mechanism and efficient log management, which significantly improves the transparency, data security and operation traceability of the system, and the specific effects are as follows:

[0170] 1. Enhance operation transparency: through aspect annotation technology, each detail in the method execution process is recorded, including parameters, return value, exception and other information, which realizes the traceability of operation process. Whether it is system exception or operation error, it can be traced back through the log to ensure the audit and monitoring of each operation.

[0171] 2. Ensure data security and compliance: real-time data verification mechanism effectively avoids illegal or non-compliant data entering the system, enhancing the security of the system. By verifying the type, format and range of data, potential injection attacks, data errors or format inconsistencies are prevented, ensuring the compliance and security of data transmission.

[0172] 3. Improve problem positioning and troubleshooting efficiency: Efficient log management mechanism ensures that operation logs, exception information, performance data, etc. can be efficiently stored and retrieved. When the system fails or abnormity, operation and maintenance personnel can quickly find the root cause of the problem through the log, greatly improving the efficiency and accuracy of fault handling, reducing system downtime.

[0173] 4. System optimization and performance improvement: Through continuous monitoring and data recording, the system can analyze performance bottlenecks, make optimization adjustments, reduce resource waste and unnecessary performance overhead. The scalability and flexibility of the system are also enhanced.

[0174] 5. Improve the reliability and stability of the system: The invention further improves the reliability and stability of the system through comprehensive monitoring and management of operations, data and logs. Data verification ensures the legality of input data, operation logs ensure the traceability of system state, and real-time monitoring helps to discover potential abnormalities or faults in time, thereby enhancing the overall stability of the system.

[0175] In another aspect, referring to Figure 5 , the embodiment of the present application provides a data operation monitoring system, comprising:

[0176] The first module 510 is configured to add a log recording aspect in the target program through aspect annotation; wherein the target program is used to execute at least one data operation task;

[0177] The second module 520 is configured to record the calling information corresponding to the target program through the log recording aspect if the target program is called; wherein the calling information includes calling time, calling parameters and calling executor;

[0178] The third module 530 is configured to record the operation result information corresponding to the target program through the log recording aspect if the target program is executed; wherein the operation result information includes execution result or exception information;

[0179] The fourth module 540 is configured to monitor the data operation based on the operation result information and the calling information.

[0180] It can be seen that the contents in the above method embodiments are applicable to the system embodiments, the system embodiments specifically realize the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0181] Referring to Figure 6 , the embodiment of the present application provides a data operation monitoring device, comprising:

[0182] At least one processor 610;

[0183] at least one memory 620, configured to store at least one program;

[0184] The at least one program, when executed by the at least one processor 610, enables the at least one processor 610 to implement the data operation monitoring method.

[0185] Similarly, the contents in the above method embodiments are all applicable to the present device embodiments, the present device embodiments specifically implement the functions same as those of the above method embodiments, and achieve the beneficial effects same as those of the above method embodiments.

[0186] The embodiment of the present application further provides a computer readable storage medium, wherein a program executable by a processor is stored, and the program executable by the processor is used for executing the above data operation monitoring method when executed by the processor.

[0187] Similarly, the contents in the above method embodiments are all applicable to the present device embodiments, the present device embodiments specifically implement the functions same as those of the above method embodiments, and achieve the beneficial effects same as those of the above method embodiments.

[0188] In some alternative embodiments, the functions / operations mentioned in the block diagram can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially simultaneously or the blocks can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flowcharts of the present application are provided by way of example, and the purpose is to provide a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are independently executed.

[0189] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be appreciated that detailed discussion of the actual implementation of each module is not necessary to an understanding of the application. Rather, the actual implementation of the modules, in light of the attributes, functions and internal relationships of the various functional modules disclosed herein, will be apparent to one of ordinary skill in the art given the benefit of this disclosure. Accordingly, the present application is not limited to the specific embodiments illustrated herein, but is applicable for use in general with any device that can benefit from the functionality of the present application. It will also be appreciated that the specific concepts disclosed herein are merely illustrative of the application and that changes in implementation and design can be made without departing from the spirit of the application. The scope of the application is to be determined by the claims appended hereto, along with the full scope of equivalents to which such claims are entitled.

[0190] If the functions are implemented in software, the functions can be stored in or implemented as one or more computer program products, which can be incorporated into a computer-readable medium for use by or in connection with an apparatus, method, or system as described herein. The computer-readable medium can be a machine-readable storage device, an article of manufacture, or a computer program product. The computer-readable medium can be a transitory or non-transitory computer-readable medium. The computer-readable medium can be a computer-readable non-transitory storage medium. The computer-readable non-transitory storage medium can be a machine-readable storage device. The computer-readable non-transitory storage medium can include one or more of: a portable computer diskette, a hard disk, a RAM, a ROM, a FLASH memory, a compact disc-ROM (CD-ROM), and so on. The computer program product can include a computer program, which can be implemented in program code (e.g., in the form of one or more programs), and which can be executed by one or more processors.

[0191] Logic and / or steps represented in flow diagrams and / or described herein, for example, can be embodied in computer-readable instructions, which can be used to cause one or more computer processors to perform the recited operations. The computer-readable instructions can be stored on one or more computer-readable media, which can include any available memory or storage device such as volatile memory, non-volatile memory, or a combination of the two. The computer-readable instructions can be part of, produced by, or used in an operating system or a stand-alone application executed on or by the computer system. The computer-readable instructions might also be distributed over networks, for example, for read and execution by one or more computer processors.

[0192] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0193] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above described embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable

[0194] In the above description of the present application, reference has been made to descriptive terms such as "one embodiment," "another embodiment," "some embodiments," etc. Such descriptive terms mean that a particular feature, structure, material or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily referring to the same embodiment. Further, when a particular feature, structure, material or characteristic is described in connection with any one or more embodiments, it is submitted that it is within the purview of the inventor(s) to effect such feature, structure, material or characteristic in connection with any other or all embodiments.

[0195] While the embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, combinations, and alterations can be made to the embodiments without departing from the spirit and scope of the present application, which is defined by the following claims and their equivalents.

[0196] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope of the present application defined by the claims.

Claims

1. A method of monitoring data operations, characterized by, The method comprises the following steps: adding a logging aspect in a target program through an aspect annotation, wherein the target program is used to perform a task of at least one data operation; recording calling information corresponding to the target program through the logging aspect if the target program is called, wherein the calling information comprises calling time, calling parameters, and calling executor; recording operation result information corresponding to the target program through the logging aspect if the target program is executed, wherein the operation result information comprises execution result or exception information; monitoring the data operation based on the operation result information and the calling information.

2. The data operation monitoring method of claim 1, wherein, The method further comprises: adding an annotation in the target program to trigger a data verification aspect before the target program is executed, so as to verify parameters of the target program; recording and returning error information if illegal data is found through the data verification aspect, and the target program is not executed.

3. The data operation monitoring method of claim 1, wherein, The method further comprises: obtaining to-be-verified parameters of the target program through a surrounding notification connection point; determining a verification strategy according to a data type of the to-be-verified parameters; performing recursive checking or data verification on the to-be-verified parameters through a data verification framework if the to-be-verified parameters are nested structures; recording data exception information if data exception is found through data verification.

4. The data operation monitoring method of claim 1, wherein, The method further comprises: adding an exception throwing post-notification annotation in the target program to capture exception information when the target program throws an exception; extracting type, location, and stack information in the exception information, and storing the exception information according to a preset exception processing strategy, recording the exception information according to a level, or providing recovery measures.

5. The data operation monitoring method of claim 1, wherein, The logging aspect comprises pre-notification, surrounding notification, and post-return notification. The logging aspect records calling information corresponding to the target program, comprising: recording calling parameters before the target program is executed through pre-notification; recording time stamps before and after the target program is executed through surrounding notification, and determining calling time; obtaining calling executor according to preset parameters through surrounding notification; converting the calling information into structured log data according to a preset data format; The logging aspect records operation result information corresponding to the target program, comprising: recording operation result information of the target program through post-return notification; converting the operation result information into structured log data according to a preset data format; calling a log framework to output the structured log data.

6. The data operation monitoring method of claim 1, wherein, The method further comprises: storing the calling information and the operation result information in a database according to a preset data format, comprising: determining elements of a database format template according to data characteristics of the calling information and the operation result information; the elements comprise data, time, and operation related information; constructing a database structure conforming to the preset data format according to the elements, so as to store the calling information and the operation result information in the database according to the preset data format.

7. The data operation monitoring method of claim 3, wherein, If the to-be-verified parameters do not belong to nested structures, data verification is performed through the following steps: Obtaining a parameter type of the to-be-verified parameter, judging whether the parameter type is consistent with a preset parameter type; If the to-be-verified parameter belongs to a numerical value type parameter, judging whether a numerical value of the to-be-verified parameter is in a preset numerical value interval; If the to-be-verified parameter contains a string or a date, performing format verification on the string or the date.

8. A data operation monitoring system, characterized by The method comprises the following steps: A first module is configured to add a log recording aspect in a target program through an aspect annotation, wherein the target program is used to execute a task of at least one data operation; A second module is configured to record, through the log recording aspect, calling information corresponding to the target program if the target program is called, wherein the calling information comprises calling time, calling parameters and calling executors; A third module is configured to record, through the log recording aspect, operation result information corresponding to the target program if the target program is executed, wherein the operation result information comprises an execution result or exception information; A fourth module is configured to monitor the data operation based on the operation result information and the calling information.

9. A data operation monitoring apparatus characterized by comprising: The method comprises the following steps: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the data operation monitoring method according to any one of claims 1 to 7.

10. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that, The program executable by the processor is used to implement the data operation monitoring method according to any one of claims 1 to 7 when executed by the processor.

Images