Log recording method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of log recording, can be applied to business system platforms of medical health, financial science and technology and the like, and discloses a log recording method and device, computer equipment and a storage medium. Constructing a shared memory space for storing logs generated by different threads, and determining a unique identifier of the shared memory space; based on the thread execution strategy, constructing a parent thread and a child thread used for calling the to-be-called service; recording a parent thread log and a child thread log when the parent thread and the child thread execute the service code according to the thread execution strategy, and storing the parent thread log and the child thread log into the shared memory space according to the unique identifier; sending the parent thread log and the child thread log in the shared memory space to an asynchronous message queue for desensitization processing, and storing the parent thread log and the child thread log in a target database; therefore, the centrality and the security of log recording in a multi-thread environment can be realized.

Description

Technical Field

[0001] The present invention relates to the field of log recording technology, and in particular to a log recording method, apparatus, computer equipment, and computer-readable storage medium. Background Art

[0002] With the increasing complexity of modern software systems, multithreaded programming has become a common technique for achieving high performance and high concurrency. In a multithreaded environment, logging is an important tool for monitoring the operating status of software systems, troubleshooting issues, and optimizing performance. However, traditional logging methods have the following problems in multithreaded environments:

[0003] 1. Dispersion of log data:

[0004] In a multi-threaded environment, each thread may generate log data independently, resulting in log data being scattered across different storage locations, making it difficult to manage and analyze uniformly.

[0005] 2. Security of log data:

[0006] Log data may contain sensitive information (such as user data and passwords). Directly storing and transmitting this data can lead to security risks. Traditional logging methods generally cannot support dynamic desensitization of sensitive information and require developers to manually filter it.

[0007] In the healthcare sector, log data may contain highly sensitive data such as patients' personal information, medical records, and diagnostic results. The decentralized storage and leakage of this data can pose a serious threat to patients' privacy and security. Therefore, logging in the healthcare sector must ensure the security and privacy of sensitive information.

[0008] In the fintech sector, log data may contain sensitive data such as users' financial information, transaction records, and account balances. Distributed storage and leakage of this data can lead to financial risks and user capital losses. Therefore, logging in the fintech sector must ensure the security and privacy of sensitive information.

[0009] Based on this, how to provide a logging method, apparatus, computer device and computer-readable storage medium that can achieve centralized and secure logging in a multi-threaded environment is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0010] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a logging method, apparatus, computer device and computer-readable storage medium, aiming to solve the problem of how to achieve centralization and security of logging in a multi-threaded environment.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions:

[0012] In a first aspect, the present invention provides a log recording method, comprising:

[0013] When receiving a service call request for a service to be called from a client, constructing a shared memory space for storing logs generated by different threads and determining a unique identifier for the shared memory space; wherein the service call request includes a thread execution policy for the service to be called;

[0014] Based on the thread execution strategy, construct a parent thread and a child thread for calling the service to be called;

[0015] Recording a parent thread log and a child thread log when the parent thread and the child thread execute the business code according to the thread execution policy, and storing the parent thread log and the child thread log in the shared memory space according to the unique identifier;

[0016] When the service call request is processed, the parent thread log and the child thread log in the shared memory space are sent to the asynchronous message queue for desensitization, and the desensitized parent thread log and the child thread log are stored in the target database.

[0017] In a second aspect, the present invention provides a log recording device, comprising:

[0018] A first building module is configured to, upon receiving a service call request from a client regarding a service to be called, build a shared memory space for storing logs generated by different threads and determine a unique identifier for the shared memory space; wherein the service call request includes a thread execution policy for the service to be called;

[0019] A second building module is used to build a parent thread and a child thread for calling the service to be called based on the thread execution strategy;

[0020] A recording module, configured to record a parent thread log and a child thread log when the parent thread and the child thread execute the business code according to the thread execution policy, and store the parent thread log and the child thread log in the shared memory space according to the unique identifier;

[0021] The desensitizing module is used to send the parent thread log and the child thread log in the shared memory space to the asynchronous message queue for desensitization processing when the service call request is processed, and store the desensitized parent thread log and the child thread log in the target database.

[0022] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the logging method as described above when executing the computer program.

[0023] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program implements the logging method described above when executed by a processor.

[0024] Compared with the prior art, the present invention provides a log recording method, apparatus, computer equipment and computer-readable storage medium, wherein, when a service call request regarding a service to be called is received from a client, a shared memory space is constructed for storing logs generated by different threads, and a unique identifier of the shared memory space is determined; wherein, the service call request includes a thread execution policy of the service to be called; based on the thread execution policy, a parent thread and a child thread are constructed for calling the service to be called; the parent thread log and the child thread log are recorded when the parent thread and the child thread execute business code according to the thread execution policy, and the parent thread log and the child thread log are stored in the shared memory space according to the unique identifier; when the service call request is processed, the parent thread log and the child thread log in the shared memory space are sent to an asynchronous message queue for desensitization processing, and the desensitized parent thread log and the child thread log are stored in a target database; thereby, the present invention can achieve the centralization and security of log recording in a multi-threaded environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A schematic diagram of an application environment of a log recording method provided by an embodiment of the present invention.

[0027] Figure 2 A flowchart of a log recording method provided by one embodiment of the present invention.

[0028] Figure 3 A schematic diagram of program modules of a log recording device provided by one embodiment of the present invention.

[0029] Figure 4A schematic diagram of the structure of a computer device provided in one embodiment of the present invention.

[0030] Figure 5 Another structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0032] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0033] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0034] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0035] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0036] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0037] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0038] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0039] An embodiment of the present invention provides a log recording method that can be applied to Figure 1 In the application environment shown, the client and server communicate via a network. The client includes, but is not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, personal digital assistants (PDAs), and other computer devices. The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0040] See also Figure 2 An embodiment of the present invention provides a log recording method, wherein the method comprises the following steps:

[0041] S100: When receiving a service call request for a service to be called from a client, constructing a shared memory space for storing logs generated by different threads, and determining a unique identifier for the shared memory space; wherein the service call request includes a thread execution policy for the service to be called;

[0042] S200: Constructing a parent thread and a child thread for calling the service to be called based on the thread execution strategy;

[0043] S300, recording the parent thread log and the child thread log when the parent thread and the child thread execute the business code according to the thread execution policy, and storing the parent thread log and the child thread log in the shared memory space according to the unique identifier;

[0044] S400. When the service call request is processed, the parent thread log and the child thread log in the shared memory space are sent to the asynchronous message queue for desensitization processing, and the desensitized parent thread log and the child thread log are stored in the target database.

[0045] In specific implementation, the logging method of this embodiment achieves centralized and secure logging in a multi-threaded environment through a series of carefully designed steps. The specific analysis is as follows:

[0046] 1. Construction of shared memory space

[0047] When a service call request is received, a shared memory space is created and assigned a unique identifier. This shared memory space serves as a unified storage location for all thread logs, ensuring centralized log data. Log data generated by different threads is no longer stored separately, but is instead stored uniformly in the shared memory space, facilitating subsequent management and analysis.

[0048] 2. Unified storage of log data

[0049] Log data generated by parent and child threads when executing business code is stored in shared memory using unique identifiers. This centralized storage makes log data management more efficient, avoids the problem of log data being scattered across different locations, and facilitates subsequent log query and analysis.

[0050] 3. Dynamic desensitization

[0051] Once the service call request is processed, the log data in the shared memory space is sent to an asynchronous message queue for desensitization. Desensitization is based on a pre-set dynamic desensitization policy, automatically identifying and processing sensitive information in the logs, such as user data and passwords. This dynamic desensitization mechanism ensures the security of log data during storage and transmission, preventing the leakage of sensitive information.

[0052] 4. Persistent storage of the target database

[0053] The desensitized log data is stored in the target database. The target database typically has higher security and access control mechanisms, further protecting the security of the log data. Furthermore, by storing log data in a database, you can leverage the database's encryption, backup, and recovery capabilities to further enhance data security.

[0054] The above approach significantly improves the centralization and security of logging. The use of shared memory ensures centralized storage of log data, avoiding data fragmentation and improving log management efficiency and analysis convenience. Dynamic desensitization and persistent storage in the target database technically guarantee the security of log data, preventing the leakage of sensitive information and meeting the compliance requirements of sectors with high data security requirements, such as healthcare and fintech. This comprehensive technical solution not only improves logging efficiency but also enhances the overall security of the system.

[0055] It is understandable that the log recording method provided by the embodiment of the present invention can be applied to log recording scenarios related to the medical and health field. The following is a specific example:

[0056] Background: In the healthcare sector, log data may contain highly sensitive data such as patients' personal information, medical records, and diagnostic results. The storage and transmission of this data must strictly comply with data protection regulations (such as HIPAA) to ensure patient privacy and data security.

[0057] Specific examples:

[0058] 1. Service call request

[0059] Consider a service call request in a medical information system: "Query patient medical records." This request includes a thread execution strategy. For example, the core logic needs to be processed in the main thread while the patient's examination report needs to be asynchronously loaded in a child thread.

[0060] 2. Build a shared memory space

[0061] When the system receives a request to query patient medical records, it builds a shared memory space to store the logs generated by the main thread and child threads. At the same time, it generates a unique identifier (such as a UUID) to identify this shared memory space.

[0062] 3. Build parent thread and child thread

[0063] According to the thread execution strategy, the system creates a parent thread to handle core logic, such as verifying user permissions and retrieving basic medical record information. At the same time, a child thread is created to asynchronously load the patient's examination report.

[0064] 4. Record logs and store them in shared memory space

[0065] The parent thread logs a message such as "User [user ID] queries the medical records of patient [patient ID]." The child thread logs a message such as "Loading of examination report for patient [patient ID] completed." These logs contain thread context information (such as the thread ID and call stack) and are stored in shared memory using unique identifiers.

[0066] 5. Desensitization and persistent storage

[0067] Once the service call request is processed, the system sends the log data in the shared memory space to an asynchronous message queue for desensitization. Desensitization strategies may include replacing patient IDs with encrypted values ​​or hiding sensitive information in medical records. Desensitized log data is ultimately stored in the target database, ensuring data security and compliance.

[0068] Technical effects:

[0069] Centralization: Log data of all threads is stored in a shared memory space, facilitating unified management and analysis;

[0070] Security: Desensitization ensures that log data does not contain sensitive information, meeting the requirements of regulations such as HIPAA.

[0071] It is understandable that the logging method provided in the embodiment of the present invention can also be applied to logging scenarios related to the financial technology field. The following is a specific example:

[0072] Background: In the fintech sector, log data may contain sensitive data such as users' financial information, transaction records, and account balances. The storage and transmission of this data must strictly adhere to financial regulatory requirements (such as GDPR and PCI DSS) to ensure data security and compliance.

[0073] Specific examples:

[0074] 1. Service call request

[0075] Consider a service call request in a fintech system: "Process a user transfer transaction." This request includes a thread execution strategy. For example, transaction verification logic needs to be processed in the main thread while transaction logging needs to be asynchronously recorded in a child thread.

[0076] 2. Build a shared memory space

[0077] When the system receives a request to process a user transfer transaction, it creates a shared memory space to store the logs generated by the main thread and child threads. At the same time, it generates a unique identifier (such as a UUID) to identify this shared memory space.

[0078] 3. Build parent thread and child thread

[0079] Based on the thread execution strategy, the system creates a parent thread to handle core logic, such as verifying user identity, checking account balances, and performing transfer operations. At the same time, a child thread is created to asynchronously record transaction logs.

[0080] 4. Record logs and store them in shared memory space

[0081] The parent thread logs a message such as "User [user ID] initiated a transfer transaction, amount [amount], from account [source account ID] to account [destination account ID]." The child thread logs a message such as "Transaction [transaction ID] logging completed." These logs contain thread context information (such as the thread ID and call stack) and are stored in shared memory using unique identifiers.

[0082] 5. Desensitization and persistent storage (S400):

[0083] Once the service call request is processed, the system sends the log data in shared memory to an asynchronous message queue for desensitization. Desensitization strategies may include replacing user and account IDs with encrypted values ​​and hiding sensitive information such as transaction amounts. Desensitized log data is ultimately stored in the target database to ensure data security and compliance.

[0084] Technical effects:

[0085] Centralization: Log data of all threads are stored in a shared memory space, facilitating unified management and analysis.

[0086] Security: Through desensitization processing, we ensure that log data does not contain sensitive information and meet the requirements of regulations such as GDPR and PCI DSS.

[0087] Through the specific application examples in the above two fields, it can be seen that the logging method of the present invention can effectively solve the centralization and security problems of logging in a multi-threaded environment, and is particularly suitable for the medical health and financial technology fields with high requirements for data security and compliance.

[0088] Furthermore, in one embodiment, the logging method, wherein, upon receiving a service call request for a service to be called from a client, constructing a shared memory space for storing logs generated by different threads and determining a unique identifier for the shared memory space, specifically comprises the steps of:

[0089] Receive a service call request from a client regarding a service to be called;

[0090] Parsing the service call request to extract the service identifier and thread execution strategy of the service to be called;

[0091] estimating the amount of logs to be generated according to the service identifier and the thread execution strategy, and constructing a shared memory space for storing logs generated by different threads based on the amount of logs;

[0092] A distributed unique identifier generation algorithm is used to generate a unique identifier for the shared memory space.

[0093] Furthermore, the logging method, wherein the estimating the amount of logs to be generated according to the service identifier and the thread execution strategy, and constructing a shared memory space for storing logs generated by different threads based on the log amount, specifically comprises the steps of:

[0094] According to the service identifier, obtaining historical log data of the service to be called;

[0095] Generate a log volume estimation result of the service call request using a pre-trained log volume estimation model according to the historical log data and the thread execution policy;

[0096] According to the log amount estimation result, a shared memory space is constructed for storing logs generated by different threads.

[0097] During specific implementation, the specific implementation process of the steps in this embodiment is roughly as follows:

[0098] 1. Receive and parse service call requests:

[0099] Receive service call requests sent by the client through the network interface.

[0100] Use a parser (such as a JSON parser or XML parser) to parse the request content and extract key information, including service identifier (such as service name or service ID) and thread execution policy (such as thread pool size, thread priority, asynchronous execution, etc.).

[0101] The extracted information is verified to ensure the validity of the service identification and thread execution policy.

[0102] 2. Obtain historical log data and estimate log volume:

[0103] According to the service identifier, the historical log data of the service to be called is queried from the log database or log storage system.

[0104] Extract features from historical log data, such as average log volume, log generation rate, log type distribution, etc.

[0105] The extracted features and thread execution strategies are input into a pre-trained log volume estimation model (such as a regression model based on machine learning) to generate a log volume estimation result for this service call request.

[0106] 3. Build a shared memory space and generate a unique identifier:

[0107] Determine the size of the shared memory space based on the log volume estimation result.

[0108] Use the memory management API (such as mmap or malloc) provided by the software system to create a shared memory space of the determined size, and initialize the management structure of the shared memory space (such as metadata of the memory pool).

[0109] A distributed unique identifier generation algorithm (such as UUID or Snowflake algorithm) is used to generate a unique identifier for the shared memory space, ensuring that each shared memory space has a globally unique identifier in a distributed system.

[0110] Through the above process, this embodiment can construct a shared memory space for storing logs generated by different threads and determine a unique identifier of the shared memory space.

[0111] Furthermore, in one embodiment, the logging method, wherein the step of constructing a parent thread and a child thread for calling the service to be called based on the thread execution strategy, specifically comprises the steps of:

[0112] Parsing the thread execution policy and extracting thread configuration information;

[0113] Creating a parent thread and at least one child thread according to the thread configuration information;

[0114] Binding the logging locations of the parent thread and the child thread to the unique identifier of the shared memory space.

[0115] During specific implementation, the specific implementation process of the steps in this embodiment is roughly as follows:

[0116] 1. Analyze thread execution strategy:

[0117] After receiving a service call request, the system parses the thread execution policy included in the request. Thread execution policies are typically stored in configuration files, JSON objects, or XML formats. The system uses appropriate parsing tools (such as JSON or XML parsers) to extract thread configuration information, ensuring that subsequent thread creation and management are performed according to the pre-set policy.

[0118] 2. Create parent thread and child thread:

[0119] According to the parsed thread configuration information, the system creates a parent thread (main thread) and at least one child thread (asynchronous thread).

[0120] The parent thread is responsible for initializing the main logic of calling the service to be called, such as verifying user permissions and obtaining necessary resources.

[0121] Child threads are responsible for executing time-consuming or parallelizable tasks, such as data loading and network requests, according to the configuration in the thread execution policy.

[0122] The system manages the life cycle of threads through the thread pool, ensuring that the creation and destruction of threads comply with the best practices of resource management.

[0123] 3. Bind the logging location to the shared memory space:

[0124] When the parent and child threads start, the system binds their logging locations to unique identifiers in the shared memory space.

[0125] Specifically, the system assigns a logger instance to each thread and uses a unique identifier in the shared memory space as the logger's context information. When a thread generates log data, the logger automatically writes the log data to the corresponding area in the shared memory space and appends thread context information (such as the thread ID and call stack information) to the log data for subsequent log analysis and tracing.

[0126] Through the above process, this embodiment can flexibly create and manage parent threads and child threads according to the thread execution policy, and ensure that their log record locations are correctly bound to the shared memory space, providing a basis for subsequent centralized log management and analysis.

[0127] Furthermore, in one embodiment, the logging method, wherein the recording of the parent thread log and the child thread log when the parent thread and the child thread execute the business code according to the thread execution policy, and storing the parent thread log and the child thread log in the shared memory space according to the unique identifier, specifically comprises the steps of:

[0128] Parsing the thread execution policy and extracting thread execution information;

[0129] Recording the parent thread log and the child thread log when the parent thread and the child thread execute the business code according to the thread execution information;

[0130] The parent thread log and the child thread log are formatted, and the formatted parent thread log and the child thread log are stored in the shared memory space according to the unique identifier.

[0131] During specific implementation, the specific implementation process of the steps in this embodiment is roughly as follows:

[0132] 1. Analyze thread execution strategy:

[0133] After receiving the service call request, the system parses the thread execution strategy carried in the request, extracts the thread execution information, and ensures that subsequent thread execution and logging can be carried out according to the preset strategy.

[0134] 2. Record the logs of the parent thread and child thread:

[0135] Based on the parsed thread execution information, the parent and child threads log the business code through their assigned loggers. The loggers automatically capture the thread context information (such as thread ID, call stack information, execution timestamp, etc.) and append this information to the log entries.

[0136] 3. Format log data:

[0137] Before the logger writes the log data to the shared memory space, it formats the log data. Formatting includes converting the log data into a unified format (such as JSON or XML) and ensuring that the log entry contains all necessary information (such as thread ID, timestamp, log level, log message, etc.).

[0138] 4. Store logs in shared memory space:

[0139] The formatted log data is stored in a shared memory space using a unique shared memory space identifier. The shared memory space can be a thread-safe data structure used to store log data from different threads. Each log entry is appended to its corresponding region in the shared memory space for subsequent processing and analysis.

[0140] 5. Monitor the usage of shared memory space (optional):

[0141] The system can set up a monitoring mechanism to monitor the usage of shared memory space in real time, including memory usage, log writing speed, etc. If memory usage approaches the threshold, the system can automatically trigger a log backup operation, backing up part of the log data to persistent storage (such as a disk file or distributed storage system) to free up memory space and ensure the integrity of the log data.

[0142] Through the above process, this embodiment can flexibly record the logs of the parent thread and the child thread according to the thread execution policy, and ensure that the log data is stored in the shared memory space in a unified format, providing a basis for subsequent log processing and analysis.

[0143] Furthermore, in one embodiment, the logging method, wherein when the service call request is processed, the parent thread log and the child thread log in the shared memory space are sent to an asynchronous message queue for desensitization processing, and the desensitized parent thread log and the child thread log are stored in a target database, specifically comprises the steps of:

[0144] Real-time monitoring of the processing progress of the service call request;

[0145] When it is monitored that the service call request is processed, extracting the parent thread log and the child thread log from the shared memory space, and sending the extracted parent thread log and the child thread log to an asynchronous message queue;

[0146] According to a preset dynamic desensitization strategy, desensitizing the parent thread log and the child thread log in the asynchronous message queue;

[0147] The desensitized parent thread log and the child thread log are persistently stored in the target database.

[0148] Furthermore, the logging method, wherein the desensitizing processing of the parent thread log and the child thread log in the asynchronous message queue according to a preset dynamic desensitization strategy, specifically comprises the steps of:

[0149] Determine a dynamic desensitization strategy in advance based on the security requirements of the service to be called;

[0150] When the parent thread log and the child thread log are detected in the asynchronous message queue, the parent thread log and the child thread log are parsed to obtain sensitive fields that need to be desensitized;

[0151] According to the dynamic desensitization strategy, a corresponding desensitization algorithm is executed on the sensitive field to complete the desensitization processing of the parent thread log and the child thread log in the asynchronous message queue.

[0152] During specific implementation, the specific implementation process of the steps in this embodiment is roughly as follows:

[0153] 1. Real-time monitoring of the processing progress of service call requests:

[0154] The system uses a monitoring mechanism to monitor the progress of service call requests in real time. This can be achieved by setting a status flag in the parent thread or using thread synchronization mechanisms. When the business logic of all relevant threads (including parent and child threads) is executed, the status flag is set to "Complete", triggering the subsequent log processing process.

[0155] 2. Extract logs from shared memory space and send them to asynchronous message queue:

[0156] When a service call request is detected to have been processed, the system extracts the parent and child thread logs from shared memory. This log data is packaged into messages and sent to an asynchronous message queue (such as RabbitMQ or Kafka). The use of message queues decouples log generation and processing, improving system responsiveness and reliability.

[0157] 3. Parse log data and obtain sensitive fields:

[0158] On the consumer side of the asynchronous message queue, the system parses the received log data. This parsing process involves converting the log data from a storage format (such as JSON or XML) into an actionable structure and extracting sensitive fields. Sensitive fields can be identified using a pre-defined regular expression or a list of field names. For example, the system might identify fields such as user IDs, passwords, and credit card numbers in the logs as sensitive information.

[0159] 4. Execute the desensitization algorithm on sensitive fields according to the dynamic desensitization strategy:

[0160] The system applies a corresponding desensitization algorithm to sensitive fields based on a pre-set dynamic desensitization policy. Dynamic desensitization policies can be flexibly configured to meet different business needs and security requirements, supporting a variety of desensitization algorithms, such as obfuscation (replacing sensitive information with "..."), encryption (encrypting sensitive information using an encryption algorithm), or hashing (replacing sensitive information with a hash value). The selection and application of desensitization algorithms can be dynamically adjusted through configuration files or the management interface without requiring code modifications.

[0161] 5. Persistently store the desensitized logs in the target database:

[0162] After desensitization is complete, the system persistently stores the desensitized log data in a target database. This target database can be a relational database (such as MySQL or PostgreSQL), a NoSQL database (such as MongoDB), or a distributed database (such as HBase). During storage, the system records the log's storage location and timestamp and generates a unique log ID for subsequent log queries and audits. The system also supports automatic backup mechanisms to ensure the reliability of log data.

[0163] Through the above process, this embodiment can monitor the processing progress of service call requests in real time and extract log data from the shared memory space after the request processing is completed. The log data is sent to the asynchronous message queue for desensitization processing, and the desensitized log data is ultimately stored persistently in the target database. This implementation method not only improves the efficiency and security of log processing, but also supports flexible dynamic desensitization policy configuration to meet the needs of different business scenarios.

[0164] As can be seen from the above method embodiments, the log recording method provided by the present invention includes: when a service call request for a service to be called is received from a client, a shared memory space is constructed for storing logs generated by different threads, and a unique identifier of the shared memory space is determined; wherein, the service call request includes a thread execution policy for the service to be called; based on the thread execution policy, a parent thread and a child thread are constructed for calling the service to be called; the parent thread log and the child thread log are recorded when the parent thread and the child thread execute the business code according to the thread execution policy, and according to the unique identifier, the parent thread log and the child thread log are stored in the shared memory space; when the service call request is processed, the parent thread log and the child thread log in the shared memory space are sent to an asynchronous message queue for desensitization processing, and the desensitized parent thread log and the child thread log are stored in a target database. In this way, the method of the present invention can achieve the centralization and security of log recording in a multi-threaded environment.

[0165] It should be understood that although the present application provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative work, and these operation steps are not necessarily performed in the order of the embodiments or flowcharts. The order of steps listed in the embodiments or flowcharts is only one way of executing the steps among many steps and does not represent the only execution order. It should be noted that there is not necessarily a certain order between the above steps. Those of ordinary skill in the art can understand from the description of the embodiments of the present invention that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or they may be executed in an interchangeable manner, etc. Moreover, at least a portion of the steps in the embodiments or flowcharts may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but may be executed in turn, alternately or synchronously with other steps or at least a portion of the sub-steps or stages of other steps.

[0166] Based on the above method embodiment, please refer to Figure 3Another embodiment of the present invention further provides a log recording device, wherein the device includes:

[0167] The first construction module 11 is configured to, upon receiving a service call request from a client regarding a service to be called, construct a shared memory space for storing logs generated by different threads and determine a unique identifier for the shared memory space; wherein the service call request includes a thread execution policy for the service to be called;

[0168] A second construction module 12 is configured to construct a parent thread and a child thread for calling the service to be called based on the thread execution policy;

[0169] A recording module 13 is used to record the parent thread log and the child thread log when the parent thread and the child thread execute the business code according to the thread execution policy, and store the parent thread log and the child thread log in the shared memory space according to the unique identifier;

[0170] The desensitizing module 14 is used to send the parent thread log and the child thread log in the shared memory space to the asynchronous message queue for desensitization processing when the service call request is processed, and store the desensitized parent thread log and the child thread log in the target database.

[0171] Furthermore, in one embodiment, the logging device, when receiving a service call request for a service to be called from a client, constructs a shared memory space for storing logs generated by different threads and determines a unique identifier for the shared memory space, specifically comprising:

[0172] Receive a service call request from a client regarding a service to be called;

[0173] Parsing the service call request to extract the service identifier and thread execution strategy of the service to be called;

[0174] estimating the amount of logs to be generated according to the service identifier and the thread execution strategy, and constructing a shared memory space for storing logs generated by different threads based on the amount of logs;

[0175] A distributed unique identifier generation algorithm is used to generate a unique identifier for the shared memory space.

[0176] Furthermore, the logging device, wherein the estimating the amount of logs to be generated according to the service identifier and the thread execution strategy, and constructing a shared memory space for storing logs generated by different threads based on the amount of logs, specifically includes:

[0177] According to the service identifier, obtaining historical log data of the service to be called;

[0178] Generate a log volume estimation result of the service call request using a pre-trained log volume estimation model according to the historical log data and the thread execution policy;

[0179] According to the log amount estimation result, a shared memory space is constructed for storing logs generated by different threads.

[0180] Furthermore, in one embodiment, the logging device, wherein the step of constructing a parent thread and a child thread for calling the service to be called based on the thread execution policy, specifically includes:

[0181] Parsing the thread execution policy and extracting thread configuration information;

[0182] Creating a parent thread and at least one child thread according to the thread configuration information;

[0183] Binding the logging locations of the parent thread and the child thread to the unique identifier of the shared memory space.

[0184] Furthermore, in one embodiment, the logging device, wherein the recording of the parent thread log and the child thread log when the parent thread and the child thread execute the service code according to the thread execution policy, and storing the parent thread log and the child thread log in the shared memory space according to the unique identifier, specifically includes:

[0185] Parsing the thread execution policy and extracting thread execution information;

[0186] Recording the parent thread log and the child thread log when the parent thread and the child thread execute the business code according to the thread execution information;

[0187] The parent thread log and the child thread log are formatted, and the formatted parent thread log and the child thread log are stored in the shared memory space according to the unique identifier.

[0188] Furthermore, in one embodiment, the logging device, wherein when the service call request is processed, sends the parent thread log and the child thread log in the shared memory space to an asynchronous message queue for desensitization processing, and stores the desensitized parent thread log and the child thread log in a target database, specifically includes:

[0189] Real-time monitoring of the processing progress of the service call request;

[0190] When it is monitored that the service call request is processed, extracting the parent thread log and the child thread log from the shared memory space, and sending the extracted parent thread log and the child thread log to an asynchronous message queue;

[0191] According to a preset dynamic desensitization strategy, desensitizing the parent thread log and the child thread log in the asynchronous message queue;

[0192] The desensitized parent thread log and the child thread log are persistently stored in the target database.

[0193] Furthermore, the log recording device, wherein the desensitizing processing of the parent thread log and the child thread log in the asynchronous message queue according to the preset dynamic desensitization strategy specifically includes:

[0194] Determine a dynamic desensitization strategy in advance based on the security requirements of the service to be called;

[0195] When the parent thread log and the child thread log are detected in the asynchronous message queue, the parent thread log and the child thread log are parsed to obtain sensitive fields that need to be desensitized;

[0196] According to the dynamic desensitization strategy, a corresponding desensitization algorithm is executed on the sensitive field to complete the desensitization processing of the parent thread log and the child thread log in the asynchronous message queue.

[0197] It should be noted that, in the embodiment of the device of the present invention, the information interaction, execution process and other contents between the above modules are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the aforementioned method embodiment part and will not be repeated here.

[0198] Based on the above method embodiment, another embodiment of the present invention further provides a computer device, which can be a server, and its internal structure diagram can be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the functions or steps on the server side of the logging method in any of the above method embodiments are implemented.

[0199] Based on the above method embodiment, another embodiment of the present invention further provides a computer device, which can be a client, and its internal structure diagram can be as follows: Figure 5 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the functions or steps of the client side of the logging method in any of the above method embodiments are implemented.

[0200] Those skilled in the art will understand that Figure 4 and Figure 5 The structural diagram shown in the figure is only a schematic diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more components than shown in the figure, or combine certain components, or have a different component arrangement.

[0201] The processor referred to herein may be a CPU, other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or any conventional processor, etc.

[0202] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory can be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium can be a hard disk of the computer device, and in other embodiments, it can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Furthermore, the memory can also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program. The memory can also be used to temporarily store data that has been output or is about to be output.

[0203] Based on the above method embodiments, another embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the logging method described in any of the above method embodiments. The computer-readable storage medium can be either non-volatile or volatile.

[0204] It should be noted that the above-mentioned functions or steps that can be implemented by computer-readable storage media or computer devices, and the technical effects brought about by the functions / steps, can be found in the relevant descriptions in the aforementioned method embodiments. To avoid repetition, they will not be described one by one here.

[0205] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM). The disclosed memory components or memories of the operating environments described herein are intended to comprise one or more of these and / or any other suitable types of memory.

[0206] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, in the embodiment of the device of the present invention, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual application, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0207] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0208] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0209] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0210] It should be noted that if software tools or components other than those of the Company appear in the embodiments of this application, they are merely for illustration and do not represent actual use. The above embodiments are intended only to illustrate the technical solutions of the present invention, not to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A logging method, characterized in that: include: When receiving a service call request for a service to be called from a client, constructing a shared memory space for storing logs generated by different threads and determining a unique identifier for the shared memory space; wherein the service call request includes a thread execution policy for the service to be called; Based on the thread execution strategy, construct a parent thread and a child thread for calling the service to be called; Recording a parent thread log and a child thread log when the parent thread and the child thread execute the business code according to the thread execution policy, and storing the parent thread log and the child thread log in the shared memory space according to the unique identifier; When the service call request is processed, the parent thread log and the child thread log in the shared memory space are sent to the asynchronous message queue for desensitization, and the desensitized parent thread log and the child thread log are stored in the target database.

2. The logging method according to claim 1, wherein: When receiving a service call request for a service to be called from a client, constructing a shared memory space for storing logs generated by different threads and determining a unique identifier of the shared memory space includes: Receive a service call request from a client regarding a service to be called; Parsing the service call request to extract the service identifier and thread execution strategy of the service to be called; estimating the amount of logs to be generated according to the service identifier and the thread execution strategy, and constructing a shared memory space for storing logs generated by different threads based on the amount of logs; A distributed unique identifier generation algorithm is used to generate a unique identifier for the shared memory space.

3. The log recording method according to claim 2, characterized in that: The estimating the amount of logs to be generated according to the service identifier and the thread execution strategy, and constructing a shared memory space for storing logs generated by different threads based on the amount of logs, includes: According to the service identifier, obtaining historical log data of the service to be called; Generate a log volume estimation result of the service call request using a pre-trained log volume estimation model according to the historical log data and the thread execution policy; According to the log amount estimation result, a shared memory space is constructed for storing logs generated by different threads.

4. The logging method according to claim 1, wherein: The step of constructing a parent thread and a child thread for calling the service to be called based on the thread execution strategy includes: Parsing the thread execution policy and extracting thread configuration information; Creating a parent thread and at least one child thread according to the thread configuration information; Binding the logging locations of the parent thread and the child thread to the unique identifier of the shared memory space.

5. The log recording method according to claim 1, wherein: The recording of the parent thread log and the child thread log when the parent thread and the child thread execute the business code according to the thread execution policy, and storing the parent thread log and the child thread log in the shared memory space according to the unique identifier, includes: Parsing the thread execution policy and extracting thread execution information; Recording the parent thread log and the child thread log when the parent thread and the child thread execute the business code according to the thread execution information; The parent thread log and the child thread log are formatted, and the formatted parent thread log and the child thread log are stored in the shared memory space according to the unique identifier.

6. The logging method according to claim 1, wherein: When the service call request is processed, the parent thread log and the child thread log in the shared memory space are sent to the asynchronous message queue for desensitization processing, and the desensitized parent thread log and the child thread log are stored in the target database, including: Real-time monitoring of the processing progress of the service call request; When it is monitored that the service call request is processed, extracting the parent thread log and the child thread log from the shared memory space, and sending the extracted parent thread log and the child thread log to an asynchronous message queue; According to a preset dynamic desensitization strategy, desensitizing the parent thread log and the child thread log in the asynchronous message queue; The desensitized parent thread log and the child thread log are persistently stored in the target database.

7. The log recording method according to claim 6, characterized in that: The desensitizing process of the parent thread log and the child thread log in the asynchronous message queue according to the preset dynamic desensitization strategy includes: Determine a dynamic desensitization strategy in advance based on the security requirements of the service to be called; When the parent thread log and the child thread log are detected in the asynchronous message queue, the parent thread log and the child thread log are parsed to obtain sensitive fields that need to be desensitized; According to the dynamic desensitization strategy, a corresponding desensitization algorithm is executed on the sensitive field to complete the desensitization processing of the parent thread log and the child thread log in the asynchronous message queue.

8. A log recording device, characterized in that: include: A first building module is configured to, upon receiving a service call request from a client regarding a service to be called, build a shared memory space for storing logs generated by different threads and determine a unique identifier for the shared memory space; wherein the service call request includes a thread execution policy for the service to be called; A second building module is used to build a parent thread and a child thread for calling the service to be called based on the thread execution strategy; A recording module, configured to record a parent thread log and a child thread log when the parent thread and the child thread execute the business code according to the thread execution policy, and store the parent thread log and the child thread log in the shared memory space according to the unique identifier; The desensitizing module is used to send the parent thread log and the child thread log in the shared memory space to the asynchronous message queue for desensitization processing when the service call request is processed, and store the desensitized parent thread log and the child thread log in the target database.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the logging method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the logging method according to any one of claims 1 to 7 is implemented.

Images