Data trace leaving method and device, equipment, storage medium and program product

Abstract

The invention discloses a data trace leaving method and device, equipment, a storage medium and a program product, and relates to the technical field of data processing, and the data trace leaving method comprises the steps of calling a target business method in response to a business request; intercepting execution of the target business method through a aspect-oriented programming framework, and collecting method parameters and context environment data according to a predefined annotation in the target business method, the annotation being obtained based on a preset business attribute and an operation attribute definition; and generating a data change record according to the method parameters and the context environment data. According to the method, data trace reserving is achieved through the section-oriented programming framework, the problem that an existing data trace reserving method is low in flexibility in a multi-service scene is solved, trace reserving logic and service processing are decoupled, and the flexibility of data trace reserving in the multi-service scene is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a data trace method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] With the improvement of enterprise data governance and business transparency requirements, data trace has become a core means to guarantee system traceability. The prior art usually adopts manual recording of changed data to a database, and a developer explicitly writes logic to capture pre-change data at a change of business code, and inserts the pre-change data into a separate historical record table.

[0003] However, the trace code of the prior art is deeply coupled with business logic, and the data capture logic needs to be repeatedly written in each business method, business rule changes need to be synchronously modified in multiple trace codes, and the trace code cannot be reused across modules, and is strongly dependent on manual coding by the developer, which makes it difficult to quickly respond to new business scenarios, and limits the flexible adaptation capability of the system in multiple business scenarios.

[0004] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide a data trace method, device, equipment, storage medium and program product, which aims to solve the technical problem of low flexibility of the prior art data trace method in multiple business scenarios.

[0006] To achieve the above purpose, the present application provides a data trace method, which comprises the following steps:

[0007] In response to a business request, a target business method is called;

[0008] The target business method execution is intercepted through an aspect-oriented programming framework, and method parameters and context environment data are collected according to pre-defined annotations in the target business method, the annotations being defined based on pre-set business attributes and operation attributes;

[0009] A data change record is generated according to the method parameters and the context environment data.

[0010] In an embodiment, the step of collecting method parameters and context environment data according to pre-defined annotations in the target business method comprises:

[0011] According to the annotations, method input parameter values before the target business method execution and method output parameter values after the target business method execution are collected;

[0012] Method parameters of the target business method are obtained according to the method input parameter values and the method output parameter values.

[0013] According to the annotation, context environment data of the target business method is collected.

[0014] In an embodiment, the step of generating a data change record according to the method parameter and the context environment data comprises:

[0015] According to the method parameter, a preset operation summary expression template in the annotation is parsed to obtain summary information;

[0016] According to the method parameter and the context environment data, detail information is obtained;

[0017] Based on the summary information and the detail information, a data change record is generated.

[0018] In an embodiment, before the step of obtaining the detail information according to the method parameter and the context environment data, the method further comprises:

[0019] Obtaining pre-change data and post-change data;

[0020] Comparing the difference between the pre-change data and the post-change data to generate a difference field detail;

[0021] The step of obtaining the detail information according to the method parameter and the context environment data comprises:

[0022] According to the difference field detail, the method parameter and the context environment data, the detail information is obtained.

[0023] In an embodiment, after the step of generating a data change record according to the method parameter and the context environment data, the method further comprises:

[0024] Structuring the data change record to obtain a structured operation log;

[0025] Post-processing the structured operation log through a distributed message queue to obtain a processed structured operation log;

[0026] Storing the processed structured operation log into an independent index table of a search engine database to construct a composite index, so as to enable a user to query log data.

[0027] In an embodiment, the step of post-processing the structured operation log through a distributed message queue to obtain a processed structured operation log comprises:

[0028] The structured operation log is processed by a consumer group of the distributed message queue as follows:

[0029] According to a preset field translation rule, the technical field value in the structured operation log is converted into a business semantic value;

[0030] According to a preset encryption field rule, the sensitive field value in the structured operation log is encrypted to obtain a processed structured operation log.

[0031] In addition, to achieve the above object, the present application also provides a data trace device, which comprises:

[0032] A request response module is configured to invoke a target business method in response to a business request;

[0033] A data collection module is configured to intercept execution of the target business method through an aspect-oriented programming framework, and collect method parameters and context environment data according to predefined annotations in the target business method, wherein the annotations are defined based on preset business attributes and operation attributes;

[0034] A record generation module is configured to generate a data change record according to the method parameters and the context environment data.

[0035] In addition, to achieve the above object, the present application also provides a data trace device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the above data trace method.

[0036] In addition, to achieve the above object, the present application also provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, wherein the computer program is executed by a processor to implement the steps of the above data trace method.

[0037] In addition, to achieve the above object, the present application also provides a computer program product, which comprises a computer program, wherein the computer program is executed by a processor to implement the steps of the above data trace method.

[0038] The application provides a data trace method, first, when a user initiates a service request, a target service method corresponding to the request is identified and called to provide a cut-in point for a data trace mechanism; then, the target service method is non-intrusively intercepted through an aspect-oriented programming framework, and the method parameters and context environment data during execution of the service method are accurately collected by using the service attribute and operation attribute information provided by the annotation to provide a basis for generating detailed data change records; finally, the complete records of data changes generated by the target service call are generated by using the collected method parameters and context environment data, so that subsequent data analysis and auditing are facilitated. The data trace method solves the problem of low flexibility of the existing data trace method in a multi-service scenario, avoids deep coupling between trace code and business logic, can better adapt to new service scenarios, and enhances the flexible adaptation capability of the system in the multi-service scenario. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0041] Figure 1 A flowchart provided by the first embodiment of the data trace method of the application;

[0042] Figure 2 A flowchart provided by the second embodiment of the data trace method of the application;

[0043] Figure 3 A module structure diagram of the data trace device of the embodiment of the application;

[0044] Figure 4 A device structure diagram of a hardware running environment related to the data trace method in the embodiment of the application.

[0045] The purpose implementation, functional characteristics and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the application, and are not used to limit the application.

[0047] In order to better understand the technical solutions of the application, the following will be described in detail in combination with the drawings in the specification and specific embodiments.

[0048] The main solution of the embodiment of the application is: in response to a service request, a target service method is called; the target service method execution is intercepted through an aspect-oriented programming framework, and method parameters and context environment data are collected according to a predefined annotation in the target service method, the annotation being defined based on preset business attributes and operation attributes; and a data change record is generated according to the method parameters and the context environment data.

[0049] With the increase of software size and system users, when a system problem occurs, there is no operation record, which makes fault troubleshooting more difficult, and also increases the complexity of data tracing. And with the improvement of enterprise requirements for data governance and business transparency, the application of data trace technology has developed from simple log recording to systematization and intelligentization. There are three types of implementation methods in the current mainstream scheme:

[0050] 1. Business layer manual coding: developers explicitly insert logic at the code location of data change operations to capture pre-change data and write it to a history table. The trace code needs to be written independently for each business module. Different business modules have different data change scenarios, and developers need to write trace code for each business module and each data change operation. The code reuse is very low, and the trace code is closely intertwined with the business code, which greatly increases the complexity of the business code.

[0051] 2. Database trigger: configure a trigger on a database table, use the trigger to listen to data operation events on the target table, and automatically insert pre-change and post-change records into the history table. Although the database trigger can record direct data changes, it is not comprehensive enough to record information such as business processes, operators, and operation time sequences involved in complex business scenarios.

[0052] 3. System log framework: in the Java development environment, use SLF4J and other management tools to call various methods in the business code to record operation events, and access the log system for centralized management. However, the trace rules of the system log scheme are fixed in the code, and if the log format needs to be adjusted (such as adding a transaction serial number), the code must be modified and the service must be restarted, which cannot dynamically respond to changes in audit policies.

[0053] In the present application, the target business method is called in response to a business request, and the data is collected by intercepting the execution of the business method using an aspect-oriented programming framework, and finally the data change record is generated, solving the problem of low flexibility of the existing data trace method in the multi-business scenario. Compared with the existing data trace method, the trace code is deeply coupled with the business logic, and the workload of repeatedly writing data capture logic in each business method is reduced. Developers only need to add a predefined annotation to the business method to achieve data trace, improve development efficiency and reduce maintenance cost. At the same time, due to the standardized definition of the annotation, the system can better adapt to new business scenarios, and the flexible adaptation ability of the system in the multi-business scenario is enhanced.

[0054] It should be noted that the execution subject of the present embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a data trace device, etc. capable of realizing the above functions. The present embodiment and the following embodiments will be described below with the data trace device as an example.

[0055] Based on this, the present application provides a data trace method, which is described with reference to Figure 1 , Figure 1 The flowchart of the first embodiment of the data trace method of the present application is shown in the figure.

[0056] In the present embodiment, the data trace method comprises steps S10-S30:

[0057] Step S10, in response to a business request, a target business method is called;

[0058] It should be noted that the business request refers to an operation initiated by the system through a front-end interface or interface call, etc. to trigger a specific business logic, such as a request submitted by a user to modify employee information in a human resource management system. The target business method is a function or method used by the system backend to process the business request, which defines the specific business logic, such as a business method for processing employee information modification, which can be implemented through a service-oriented architecture or a micro-service architecture, in which each business function is encapsulated into an independent service or method for easy calling.

[0059] It can be understood that when the business request is received, the system needs to accurately identify and call the corresponding business method to respond to the request, and by accurately calling the target business method, it ensures that the subsequent changes to the business data can be effectively recorded, thereby ensuring the integrity and accuracy of the data trace.

[0060] Step S20, intercepting the target business method execution through the aspect-oriented programming framework, and collecting method parameters and context environment data according to predefined annotations in the target business method, the annotations being defined based on preset business attributes and operation attributes;

[0061] It should be noted that the aspect-oriented programming framework is a programming technology framework for separating cross-cutting concerns (such as logging, transactions, etc.) from business logic, for example, Spring AOP framework. Interception of method execution means that the aspect can intervene and execute predefined logic before, during or after the execution of the target business method. The predefined annotations are annotations that developers customize according to actual business needs using the annotation definition mechanism provided by the aspect-oriented programming framework, containing business attributes (such as the business module to which it belongs, the business object ID, etc.) and operation attributes (such as operation type, operation summary, etc.).

[0062] It can be understood that in order to automatically collect key data in the execution process of the business method without affecting the business logic code. Since the annotations provide meta-information about business behavior, the aspect can understand the business semantics and operation characteristics of the current business method. By intercepting the target business method through the aspect-oriented programming framework, the method parameters and context environment data can be accurately obtained according to the preset business and operation attributes in the annotations. The automatic and standardized data trace is realized based on the annotation interception and collection method, avoiding manual insertion of data collection logic in the business code, reducing the invasiveness of the code, improving the development efficiency and the maintainability of the system.

[0063] Step S30, generating a data change record according to the method parameters and context environment data.

[0064] It should be noted that the method parameters refer to the input data used when the target business method is executed, including basic data types, objects, etc. The method parameters reflect the specific content of the business operation. The context environment data refers to the environmental information when the business method is executed, such as user identity, request source, timestamp, etc. These data provide background information for the business operation. The data change record refers to the record of data changes in the execution process of the business method, usually including operation type, operation object, operation time, data before and after change, etc.

[0065] It can be understood that the method parameters describe the specific content of the data change, and the context environment data provides the background information of the change. By comprehensively processing the method parameters and context environment data, the collected data is integrated into meaningful change records. The combination of the two comprehensively reflects the overall picture of the business operation, providing a basis for the completeness and traceability of data changes, so as to facilitate subsequent data analysis and auditing.

[0066] Exemplarily, in the employee management system, when the administrator initiates a business request to modify the department of an employee, the system calls a target business method for modifying the department of the employee. The business method is provided with a custom annotation @HrmDataTracesAnnotation, in which business attributes (such as module="employee management" and businessType="department information") and operation attributes (such as operationType=OperationTypeEnum.MODIFY) are defined. An aspect-oriented programming framework (such as Spring AOP) intercepts the execution of the business method, collects method parameters (such as the employee ID and the new department ID) and context environment data (such as the currently logged-in administrator user ID, the requested client IP address, the current timestamp, and the like). Finally, according to the collected data, a data change record is generated, which contains information such as the operation type (modification), the operation object (employee department information), the operator (administrator), and the data before and after the change (original department ID and new department ID).

[0067] In a feasible implementation, the step of collecting the method parameters and the context environment data according to the annotation predefined in the target business method comprises:

[0068] In step S201, the method parameter values before the execution of the target business method and the method parameter values after the execution of the target business method are collected according to the annotation.

[0069] It should be noted that the method parameter values refer to the specific values of the input parameters passed to the target business method before the execution of the method, which reflect the initial state of the business operation. The method parameter values refer to the result values or output parameter values returned by the method after the execution of the target business method, which reflect the execution result or final state of the business operation.

[0070] It can be understood that a business operation usually modifies the data in the system, and the method parameter values and the method parameter values represent the data states before and after the operation, respectively. By collecting the method parameter values and the method parameter values, the starting data and the ending data of the business operation can be accurately understood, so that the complete parameter information before and after the execution of the business method is obtained, so that the data change of the business operation can be recorded, and the data change before and after the execution of the method can be clearly recorded, so as to meet the requirement of data traceability.

[0071] In step S202, the method parameters of the target business method are obtained according to the method parameter values and the method parameter values.

[0072] It should be noted that the method parameter refers to the integration of the method parameter value and the method output value, which can completely describe the input and output of the business method execution process. The parameter encapsulation technology can be used to combine the method parameter value and the method output value into a method parameter.

[0073] It can be understood that the integration of the dispersed method parameter value and the method output value into a unified method parameter object can avoid frequent individual operation of each parameter in subsequent processing, more conveniently record and analyze the execution process of the business method, and more intuitively reflect the overall picture of the business operation, facilitating subsequent generation of data change records, improving the readability and maintainability of the code

[0074] In step S203, the context environment data of the target business method is collected according to the annotation.

[0075] It should be noted that the context environment data refers to the environmental information collected according to the metadata marking of the business and operation related attributes in the annotation, such as the current user identity, session information, server address, etc., which provides the background environment of the business operation.

[0076] It can be understood that the context environment information of the business operation execution is obtained, and the business operation is associated with a specific execution environment by collecting the context environment data, which provides more rich background data for the data change record, enhances the traceability and audit value of the data.

[0077] Exemplarily, in the method of the employee leave approval function of the employee management system, the business method has a custom annotation, which defines the rules for collecting the method parameter value (leave application ID, approval opinion, etc.) and the method output value (approval result status code). Before the method execution, the aspect-oriented programming framework collects the method parameter value through the interface, for example, the leave application ID is 1001 and the approval opinion is "approved". After the method execution, the method output value is collected, such as the approval result status code is 200 (indicating that the approval is successful). Then, these parameter values and output values are integrated into a method parameter object, which contains the parameter list (leave application ID, approval opinion) and the output list (approval result status code). At the same time, according to the rules defined by the annotation, the context environment data is collected, such as the current logged-in administrator user ID (assuming it is admin001), the requested client IP address (such as 192.168.1.100), the server address (such as server-a), etc.

[0078] In this embodiment, by collecting the method input parameter value and the method output parameter value respectively, integrating them into the method parameter, and collecting the context environment data at the same time, the data change record can more comprehensively reflect the details and background of the business operation, can more accurately record the data change before and after the execution of the business method, improve the completeness and accuracy of the data trace, and further enhance the adaptability and flexibility of the system in multiple business scenarios.

[0079] In a feasible implementation, the step of generating the data change record according to the method parameter and the context environment data comprises:

[0080] In step S301, the operation summary expression template preset in the annotation is parsed according to the method parameter to obtain summary information.

[0081] It should be noted that the operation summary expression template preset in the annotation is an expression template for generating operation summary information preset by the developer according to the business demand when defining the annotation, and usually includes placeholders and fixed texts, such as "user {userId} has performed {operationType} operation on {businessObject}". The summary information is a brief description of the data change related to the business operation, which is generated by replacing the specific values in the method parameter into the placeholder positions in the template, so that the user does not need to understand the specific detail data, and can quickly browse the general situation of the business operation in the log system or the audit report.

[0082] It can be understood that by parsing the preset operation summary expression template, the key data in the method parameter can be extracted to form the operation summary for quickly describing the core content of the business operation, and the user can quickly understand the subject, object and type of the operation without viewing the complex change details, which can improve the readability and usability of the log, and facilitate the operation and maintenance personnel and the audit personnel to quickly obtain the key information.

[0083] In step S302, the detail information is obtained according to the method parameter and the context environment data.

[0084] It should be noted that the detail information refers to the data for describing the business operation in detail, which usually includes the comparison of the data before and after the change, the specific content of the operation, etc., and the data mapping and integration technology can be used to convert the method parameter and the context environment data into the detail information. The detail information can provide the complete context of the operation, including the cause (such as input parameter), process (such as context change during operation) and result (such as output parameter) of the operation, which is crucial for subsequent data analysis, fault diagnosis and compliance audit.

[0085] It can be understood that by combining method parameters and contextual environment data, a change record containing rich details can be generated for in-depth analysis and auditing of specific circumstances of business operations, which can help auditors to review the legality, accuracy and completeness of transactions in detail, and meet the strict requirements of data governance and business transparency.

[0086] In step S303, a data change record is generated based on the summary information and the detail information.

[0087] It should be noted that the data change record is a complete record formed by combining the summary information and the detail information, and is usually stored in a structured data format (such as JSON or XML). Data encapsulation and serialization technologies (such as Jackson or Gson library) can be used to combine the summary information and the detail information into a data change record.

[0088] It can be understood that by integrating the operation summary and the details into a complete data change record, a complete record with both generalization and details can be formed, which is convenient for storage, transmission and query, and meets the needs of different users in different scenarios, improves the usability and flexibility of the data change record, and makes it better serve the goal of data governance and business transparency.

[0089] For example, in a document management system, when a user initiates a business request to update the content of a document, the system calls the target business method and intercepts the execution of the method through an aspect-oriented programming framework. According to the operation summary expression template defined by the annotation "user {userId} has performed {operationType} operation on document {documentId}", combined with the userId (user001), documentId (doc0001) and operationType (update) in the method parameters, the operation summary information is obtained as "user user001 has performed update operation on document doc0001". At the same time, according to the method parameters and the contextual environment data, the detail information is obtained, including the changed content of the document (such as old title: old document title, old content: old document content...), the changed content (such as new title: new document title, new content: new document content...), and the operation time (2024-07-01 15:30:00), operation IP (192.168.1.102) and other contextual data. Finally, these summary information and detail information are integrated into a data change record, which is stored in JSON format, which contains the summary field to store the operation summary information, and the details field to store the detailed change content and contextual information.

[0090] In this embodiment, the summary information is obtained by parsing the operation summary expression template in the annotation, the detail information is generated in combination with the method parameters and the context environment data, and finally the complete data change record is integrated to provide rich data change record content, which not only contains concise operation summary, but also contains detailed change details, and can better meet the requirements of data governance and business transparency. At the same time, the hierarchical recording mode improves the readability and usability of data, facilitates different users to quickly obtain the required information in different scenarios, and further improves the flexibility and adaptability of the system in multiple business scenarios.

[0091] In a feasible implementation, before the step of obtaining the detail information according to the method parameters and the context environment data, the method further includes:

[0092] In step S3021, the pre-change data and the post-change data are obtained.

[0093] It should be noted that the pre-change data refers to data whose data value is not changed before the execution of the business operation, and is usually an original data copy stored in a database or memory. The post-change data refers to data whose data value is changed after the execution of the business operation. The pre-change data and the post-change data can be read from the corresponding storage location by using database transaction snapshot or memory data snapshot technology, or can be obtained by injecting the variable values before and after the change through a standardized interface defined by the developer.

[0094] It can be understood that the complete pre-change and post-change states of the data affected by the business operation are obtained, the initial state and the final state of the data are understood, the difference between them is calculated, and the specific change content is recorded, so that the change of the data in the business operation can be accurately understood.

[0095] In step S3022, the difference between the pre-change data and the post-change data is compared to generate a difference field detail.

[0096] It should be noted that the difference between the pre-change data and the post-change data refers to the difference between the two groups of data, including the change of the field value, the change of the data structure, etc. The difference field detail refers to the detailed information of the field where the difference is located and the change thereof, and usually includes field name, old value, new value, etc. The data comparison algorithm (such as Hash algorithm, field-by-field comparison algorithm, etc.) can be used to compare the pre-change data and the post-change data to generate the difference field detail.

[0097] Additionally, it should be noted that if all the post-change data is directly recorded, a large amount of unchanged data can be included, resulting in waste of storage resources. Recording only the difference field details can ensure that only necessary change information is included in the log, improving data readability and query efficiency. For example, in an employee information update operation including multiple fields, if only the salary field has changed, the difference comparison can only record the old and new values of the salary field, without recording other unchanged fields (such as name, gender, etc.), thereby reducing the log volume.

[0098] It can be understood that by comparing the pre-change data and the post-change data, the difference field details can be generated to accurately find out the specific content of the data change, so that only the changed part is recorded in the detail information, reducing redundant data, which not only improves the accuracy of data change records, but also reduces the storage space occupation.

[0099] The step of obtaining the detail information according to the method parameters and the context environment data comprises:

[0100] In step S3023, the detail information is obtained according to the difference field details, the method parameters and the context environment data.

[0101] It should be noted that the detail information refers to the complete business operation detail record generated by combining the difference field details, the method parameters and the context environment data, which can be integrated into the detail information using data integration techniques (such as data merging algorithm, field filling technique).

[0102] It can be understood that the difference field details provide the specific content of the data change, the method parameters and the context environment data provide the operation background and input / output information of the change, and the combination of the three can clearly restore the whole process of the business operation. By integrating the difference field details, the method parameters and the context environment data, complete, comprehensive and accurate business operation detail information containing data difference details is generated, meeting the detailed requirements of data audit and analysis.

[0103] Illustratively, first, at the database level, the transaction snapshot function of the database can be used to obtain a snapshot of the current data as pre-change data before the business method is executed, and the updated data is queried again as post-change data after the business method is executed. For in-memory data (such as data in cache), a copy of the data can be created using memory replication technology to obtain pre-change data and post-change data.

[0104] Then, for simple data types (such as basic data types, strings, etc.), value comparison can be directly performed. For complex data types (such as objects, sets, etc.), recursive comparison can be used to compare fields one by one. For set type data (such as lists, maps), the size of the set can be compared first, and if the size is different, the difference is recorded directly, and if the size is the same, the elements are compared one by one. The generated difference field details can be stored in a list structure, and each element contains field name, old value, new value, etc.

[0105] Finally, the generated difference field details are passed to the detail information generator together with the method parameters and context environment data. In the generator, the difference field details are first converted to a suitable format, and then integrated with the method parameters (such as operation type, operation object ID, etc.) and context environment data (such as operation user, operation time, etc.). During integration, the details information structure can be filled according to the predefined details information structure, and finally the integrated details information of the business operation is obtained.

[0106] In this embodiment, by obtaining the pre-change data and post-change data and performing difference comparison, recording the difference field details, combining the method parameters and context environment data to obtain the details information, the redundancy information in the data change record is reduced, the storage efficiency and query performance are improved, and more accurate change details are provided, which can better reflect the actual impact of business operation on data, and enhance the accuracy and effectiveness of data trace. At the same time, it is more convenient to statistically analyze the frequency and amplitude of data changes, and further improve the data processing capability and adaptability of the system in multiple business scenarios.

[0107] The embodiment provides a data trace method, first, when a user initiates a business request, the target business method corresponding to the request is identified and called, and a cut-in point is provided for the data trace mechanism; then, the target business method is non-intrusively intercepted through an aspect-oriented programming framework, and the method parameters and context environment data during the execution of the business method are accurately collected using the business attribute and operation attribute information provided by the annotation, to provide a basis for generating detailed data change records; finally, the complete record of data change generated by the target business call is generated using the collected method parameters and context environment data, so as to facilitate subsequent data analysis and auditing. The embodiment solves the problem of low flexibility of the existing data trace method in multiple business scenarios, avoids deep coupling between trace code and business logic, can better adapt to new business scenarios, and enhances the flexible adaptation capability of the system in multiple business scenarios.

[0108] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above first embodiment can be referred to the above introduction, and will not be described in detail. On this basis, please refer to Figure 2 , Figure 2Flowchart of the second embodiment of the data trace method of the present application.

[0109] In this embodiment, after the step of generating a data change record according to the method parameters and the context environment data, the method further comprises:

[0110] In step S304, the data change record is structured to obtain a structured operation log.

[0111] It should be noted that structured processing refers to the process of converting data change records into a predefined structure and format, which usually includes data cleaning, format conversion, standardization, etc. The structured operation log refers to the log data after structured processing, which has a unified format and structure, facilitating storage, query and analysis.

[0112] It can be understood that converting the data change record into a structured operation log can eliminate inconsistencies, redundancies and errors in the data, making the data meet the storage requirements of the search engine database for subsequent storage and efficient query.

[0113] In step S305, the structured operation log is post-processed by a distributed message queue to obtain a processed structured operation log.

[0114] It should be noted that the distributed message queue refers to a distributed middleware for message transmission, such as Kafka or RabbitMQ. Post-processing refers to further processing of the structured operation log for data analysis and auditing, such as data translation, encryption, compression, storage, etc. The processed structured operation log refers to the final log data after post-processing, which has higher security and readability.

[0115] It can be understood that the consumer-producer mode of the distributed message queue is used to realize asynchronous post-processing of the structured operation log, which can decouple the processing process of the structured operation log from the business logic, avoid the influence of log processing on business response time, and improve the performance of the system and the security and readability of the log data. Since log processing is usually a relatively time-consuming operation, if the log is processed synchronously during the execution of the business method, the business response may be slow. By transmitting the log data asynchronously through the message queue, the business method can return immediately after sending the log data to the message queue, without affecting the normal execution of the business. At the same time, post-processing such as data translation and encryption at the consumer end can improve the security and readability of the log data.

[0116] In step S306, the processed structured operation log is stored in a separate index table of the search engine database to construct a composite index for user log data query.

[0117] It should be noted that the search engine database refers to a database system specially used for full-text retrieval and data analysis, such as Elasticsearch (hereinafter referred to as ES). The independent index table refers to a special index storage space created in the search engine database for structured operation logs, used to isolate different types of log data. The composite index refers to an index structure constructed according to multiple fields (such as user ID, operation time, operation type, etc.), used to improve query efficiency. Log data query refers to the operation of users obtaining log data through a query interface.

[0118] It can be understood that storing the processed structured operation logs into the search engine database can utilize its powerful full-text retrieval capability and data analysis function to meet the diversified query requirements of users for log data. According to the characteristics of high-efficiency retrieval and large-scale data storage of the search engine database, an efficient index is constructed, which can quickly respond to complex query requests, so that users can quickly query and analyze log data, and the query performance in the multi-condition combination query scenario can be further improved.

[0119] Exemplarily, in an e-commerce order management system, when a merchant initiates a business operation of modifying the order status, the system generates a data change record. After structured processing of the data change record, a structured operation log is obtained, containing fields such as operator (merchant001), operation type (modify order status), order ID (order100001), pre-change status (to be paid), post-change status (paid), operation time (2024-05-20 09:15:30), etc. Then, the structured operation log is sent to the topic “order-operation-logs” of the Kafka message queue. The Kafka consumer group (such as the log processing service) subscribes to the topic, and after receiving the message, according to the preset field translation rule, translates the order status value from the technical code (such as 0 represents to be paid, and 1 represents paid) to the business semantic value (“to be paid” and “paid”), and performs encryption processing on the sensitive fields (such as the contact information of the merchant). The processed structured operation log is stored in the datatrace_records independent index table of ES, and a composite index based on the fields of user_id, operation_time, operation_type, etc. is constructed. Users can input query conditions (such as the operator contains “merchant001” and the operation time is in May 2024) through the query interface provided by ES, to quickly retrieve the related order status change logs, to support rapid fault positioning and compliance audit.

[0120] In an implementable embodiment, the step of post-processing the structured operation log through the distributed message queue to obtain a processed structured operation log comprises:

[0121] The structured operation log is processed by the consumer group of the distributed message queue as follows:

[0122] In step S3041, the technical field value in the structured operation log is converted into a business semantic value according to a preset field translation rule.

[0123] It should be noted that asynchronous transmission refers to sending the structured operation log through the distributed message queue in a non-blocking manner to the consumer group, so that the log producer (such as a business system) and the consumer (such as a log processing service) can run independently and do not interfere with each other. The preset field translation rule refers to a rule for mapping technical field values to business semantic values according to business requirements, which is usually stored in a configuration file or a database. The technical field value refers to a field value used internally by the system and having a technical meaning, such as a status code, an enumeration value, etc. The business semantic value refers to a text description with a clear business meaning, such as "in service" and "off service".

[0124] It can be understood that since the technical field value is usually designed for the system internally, it may be difficult for business personnel to understand. Through field translation, the technical field value can be converted into a semantic value that can be understood by business personnel, improving the readability of the log. Through asynchronous transmission, it can be ensured that the log processing process does not affect the normal operation of the business system, ensuring the continuity and response speed of the business.

[0125] In step S3042, the sensitive field value in the structured operation log is encrypted according to a preset encryption field rule to obtain a processed structured operation log.

[0126] It should be noted that the preset encryption field rule refers to a rule for encrypting sensitive fields according to a security policy, including encryption algorithms, key management, etc. The sensitive field value refers to a field value containing sensitive information, such as user personal information, financial information, etc. The processed structured operation log refers to the final log data after field translation and encryption processing, which has better readability and security.

[0127] It can be understood that since the log data may contain a large amount of sensitive information, if it is not encrypted, once the log data is accessed by unauthorized access, it may cause serious security problems. Through encryption processing, the sensitive fields of the structured operation log are enhanced for security, which can protect the privacy and data security of users and avoid leakage of sensitive information.

[0128] Exemplarily, in a medical information system, when a doctor initiates a business operation of updating a patient's medical record, the system generates a structured operation log containing a technical field (such as a disease status code: status = "S101") and a sensitive field (such as a patient's ID number: id = "11010119900101001X"). The structured operation log is sent asynchronously to the topic "medical-records-logs" of the message queue through the Kafka producer. After the Kafka consumer group receives the message, according to the preset field translation rule, the disease status code "S101" is translated into the business semantic value "hypertension". At the same time, according to the preset encryption field rule, the patient's ID number is encrypted using an encryption algorithm to obtain an encrypted value, and a [ENCRYPTED] prefix is added for identification. The processed structured operation log is stored in the independent index table of Elasticsearch, and when the business personnel view the log, they can see "disease status: hypertension", while the encrypted ID number is stored and displayed in a secure manner.

[0129] In this embodiment, asynchronous post-processing is performed using a distributed message queue, the readability of the log is improved through field translation, and the security of the data is enhanced through encryption processing. While ensuring the performance of log processing, more attention is paid to the quality and security of log data, so that the system can better meet the business requirements and security compliance requirements, and the reliability of the system in multiple business scenarios is improved.

[0130] In this embodiment, through the processes of structured processing of data change records, distributed message queue post-processing, and search engine database storage, efficient processing and storage of log data are realized, the performance of the system and the queryability of log data are improved, the problem of the influence of log processing on business performance is solved, and the efficiency and flexibility of log query are greatly improved by using the full-text search and index functions of the search engine, further enhancing the applicability and practicality of the system in multiple business scenarios.

[0131] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data trace method of the present application. More forms of simple changes based on this technical concept are within the protection scope of the present application.

[0132] The present application also provides a data trace device, please refer to Figure 3 The data trace device comprises:

[0133] The request response module 10 is configured to call the target business method in response to the business request.

[0134] The data collection module 20 is configured to intercept execution of the target business method through an aspect-oriented programming framework, and collect method parameters and context environment data according to predefined annotations in the target business method, wherein the annotations are defined based on preset business attributes and operation attributes.

[0135] The record generation module 30 is configured to generate a data change record according to the method parameters and the context environment data.

[0136] Optionally, the data collection module 20 is further configured to:

[0137] According to the annotations, the method input parameter values before execution of the target business method and the method output parameter values after execution of the target business method are collected.

[0138] According to the method input parameter values and the method output parameter values, the method parameters of the target business method are obtained.

[0139] According to the annotations, the context environment data of the target business method is collected.

[0140] Optionally, the record generation module 30 is further configured to:

[0141] According to the method parameters, an operation summary expression template preset in the annotations is parsed to obtain summary information.

[0142] According to the method parameters and the context environment data, detail information is obtained.

[0143] Based on the summary information and the detail information, a data change record is generated.

[0144] Optionally, the record generation module 30 is further configured to:

[0145] The pre-change data and the post-change data are obtained.

[0146] The difference between the pre-change data and the post-change data is compared to generate a difference field detail.

[0147] According to the difference field detail, the method parameters and the context environment data, the detail information is obtained.

[0148] Optionally, the record generation module 30 is further configured to:

[0149] The data change record is subjected to structured processing to obtain a structured operation log.

[0150] The structured operation log is subjected to post-processing through a distributed message queue to obtain a processed structured operation log.

[0151] The processed structured operation log is stored into a separate index table of a search engine database, and a composite index is built for user log data query.

[0152] Optionally, the record generation module 30 is further configured to:

[0153] The structured operation log is processed by a consumer group of the distributed message queue as follows:

[0154] According to a preset field translation rule, a technical field value in the structured operation log is converted into a business semantic value.

[0155] According to a preset encryption field rule, a sensitive field value in the structured operation log is encrypted to obtain a processed structured operation log.

[0156] The data trace device provided in the present application adopts the data trace method in the above embodiments, and can solve the technical problem of low flexibility of the existing data trace method in a multi-service scenario. Compared with the prior art, the data trace device provided in the present application has the same beneficial effects as the data trace method provided in the above embodiments, and other technical features in the data trace device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0157] The present application provides a data trace device, which comprises at least one processor and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the data trace method in the first embodiment.

[0158] Reference will now be made to the drawings, and specific examples thereof will be illustrated. Figure 4 which shows a structural diagram of a data trace device suitable for implementing the embodiments of the present application. The data trace device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 4 The data trace device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0159] As Figure 4As shown, the data footprinting device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory 1002 or loaded from a storage device 1003 into a random access memory 1004. Various programs and data required for the operation of the data footprinting device are also stored in the random access memory 1004. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other by a bus 1005. An input / output interface 1006 is also connected to the bus. Generally, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data footprinting device to communicate with other devices wirelessly or by wire to exchange data. Although the data footprinting device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0160] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0161] The data footprinting device provided by the present disclosure adopts the data footprinting method in the above-mentioned embodiments, and can solve the technical problem of low flexibility of the existing data footprinting method in a multi-service scenario. Compared with the prior art, the data footprinting device provided by the present disclosure has the same beneficial effects as the data footprinting method provided by the above-mentioned embodiments, and other technical features in the data footprinting device are the same as the features disclosed in the previous embodiment method, which will not be described here.

[0162] It should be understood that parts of the present disclosure can be realized by hardware, software, firmware or a combination thereof. In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0163] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which shall be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

[0164] The present application provides a computer readable storage medium having computer readable program instructions (i.e. computer programs) stored thereon, the computer readable program instructions being used to execute the data trace method in the above embodiment.

[0165] The computer readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared or semiconductor system, system or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to: electric wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0166] The above computer readable storage medium can be contained in the data trace device; or can exist separately without being assembled into the data trace device.

[0167] The above computer readable storage medium carries one or more programs, when the one or more programs are executed by the data trace device, the data trace device: in response to a service request, calls a target service method; intercepts the execution of the target service method through an aspect-oriented programming framework, and collects method parameters and context environment data according to a predefined annotation in the target service method, the annotation being defined based on preset business attributes and operation attributes; and generates a data change record according to the method parameters and the context environment data.

[0168] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0169] The flow diagrams and the block diagrams in the drawings are meant as methodological and functional description of implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0170] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0171] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the above-mentioned data trace method, and can solve the technical problem of low flexibility of the existing data trace method in a multi-service scenario. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the data trace method provided by the above-mentioned embodiments, and will not be described here.

[0172] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the data leaving method as described above.

[0173] The computer program product provided by the application can solve the technical problem of low flexibility of the existing data leaving method in a multi-service scenario. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the data leaving method provided by the above-mentioned embodiments, and are not described here.

[0174] The above only describes some embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation made by using the content of the specification and drawings, or direct / indirect application in other related technical fields under the technical concept of the application is included in the patent protection scope of the application.

Claims

1. A data trace method, characterized in that: The data trace method includes: In response to the business request, call the target business method; intercepting the execution of the target business method through an aspect-oriented programming framework, and collecting method parameters and context data based on predefined annotations in the target business method, wherein the annotations are defined based on preset business attributes and operation attributes; Generate a data change record based on the method parameters and context data.

2. The data trace method according to claim 1, characterized in that: The step of collecting method parameters and context data according to the predefined annotations in the target business method includes: According to the annotation, the method input parameter value before the target business method is executed and the method output parameter value after the target business method is executed are collected; Obtaining method parameters of the target business method according to the method input parameter value and the method output parameter value; According to the annotation, context data of the target business method is collected.

3. The data trace method according to claim 1, characterized in that: The step of generating a data change record according to the method parameters and context data includes: According to the method parameters, the operation summary expression template preset in the annotation is parsed to obtain summary information; Obtain detailed information based on the method parameters and context data; A data change record is generated based on the summary information and the detailed information.

4. The data trace method according to claim 3, characterized in that: Before the step of obtaining detailed information according to the method parameters and context data, the method further includes: Obtain data before and after the change; Comparing the differences between the data before and after the change, and generating a detailed list of difference fields; The step of obtaining detailed information according to the method parameters and context data includes: Detailed information is obtained based on the difference field details, method parameters and context environment data.

5. The data trace method according to claim 1, wherein: After the step of generating a data change record according to the method parameters and context data, the method further includes: Structuring the data change record to obtain a structured operation log; Post-processing the structured operation log through a distributed message queue to obtain a processed structured operation log; The processed structured operation log is stored in an independent index table of a search engine database to construct a composite index for users to query log data.

6. The data trace method according to claim 5, characterized in that: The step of performing post-processing on the structured operation log through a distributed message queue to obtain a processed structured operation log includes: The structured operation log is processed as follows by the consumer group of the distributed message queue: Converting technical field values ​​in the structured operation log into business semantic values ​​according to preset field translation rules; According to the preset encryption field rules, the sensitive field values ​​in the structured operation log are encrypted to obtain a processed structured operation log.

7. A data trace device, characterized in that: The data trace device includes: The request response module is used to respond to business requests and call target business methods; a data collection module, configured to intercept the execution of the target business method through an aspect-oriented programming framework and collect method parameters and context data based on predefined annotations in the target business method, wherein the annotations are defined based on preset business attributes and operation attributes; The record generation module is used to generate a data change record based on the method parameters and context environment data.

8. A data trace device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data trace method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data trace method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the data trace method according to any one of claims 1 to 6 are implemented.

Images