Synchronous task log collection and visual retrieval method, device and equipment and medium

Abstract

The present application relates to artificial intelligence technology, disclose a kind of synchronous task log collection and visual retrieval method, comprising: in response to the start of synchronous task, create the synchronous task log object corresponding to the synchronous task;According to the synchronous task object and source system of the synchronous task, create the log index corresponding to the synchronous task log generated by the synchronous task;According to the synchronous task log object, the synchronous task log and the log index are saved in the classification list of preset database;In response to the retrieval request received, according to the retrieval request, index query is carried out on the classification list of preset database, and retrieval result is obtained.The present application also proposes a kind of synchronous task log collection and visual retrieval device, electronic equipment and storage medium.The present application can improve log collection accuracy and retrieval speed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for collecting and visualizing synchronous task logs. Background Technology

[0002] With the development of information technology, the level of social informatization is increasing, and more and more information systems are being built. Monitoring the operational status of these systems typically requires collecting system log files, analyzing and retrieving these log files to analyze related system problems.

[0003] During data synchronization or migration, due to the large volume, variety, and high frequency of access to data, abnormal situations generated during the execution of synchronization tasks are temporarily stored in local server files, which can easily lead to incomplete log information and make it difficult to obtain and maintain, resulting in low accuracy of log collection. During retrieval, the large volume of data puts a lot of pressure on the log interface, resulting in slow retrieval speed. Summary of the Invention

[0004] This invention provides a method, apparatus, and computer-readable storage medium for collecting and visually retrieving synchronous task logs, with the main purpose of solving the problems of low accuracy in log collection and slow retrieval speed.

[0005] To achieve the above objectives, the present invention provides a method for collecting and visually retrieving synchronous task logs, comprising:

[0006] In response to the start of a synchronization task, a synchronization task log object corresponding to the synchronization task is created;

[0007] Based on the synchronization task object and source system of the synchronization task, create a log index corresponding to the synchronization task log generated by the synchronization task;

[0008] Based on the synchronization task log object, the synchronization task log and the log index are stored in a category list of a preset database;

[0009] In response to a received search request, the system performs an index query on the category list of the preset database according to the search request to obtain search results.

[0010] Optionally, the step of storing the synchronization task log and the log index in a preset database category list according to the synchronization task log object includes:

[0011] Initialize the log parameters to obtain the log initialization parameters;

[0012] Based on the log initialization parameters, instantiate the synchronization task log object to obtain the target synchronization task log object;

[0013] Determine if the synchronization task log is abnormal;

[0014] When the synchronization task log is abnormal, the abnormal synchronization task log and the log index are saved in the abnormal list of the preset database according to the add log function in the target synchronization task log object.

[0015] When the synchronization task log is normal, the normal synchronization task log and the log index are saved in the normal list of the preset database according to the add log function in the target synchronization task log object.

[0016] Clear the target synchronization task log object.

[0017] Optionally, the step of saving the abnormal synchronization task log and the log index in the abnormal list of a preset database according to the add log function in the target synchronization task log object includes:

[0018] Parse the abnormal synchronization task log to obtain the target abnormal synchronization task log in a preset format;

[0019] The target abnormal synchronization task logs are classified according to application scenarios to obtain log types;

[0020] Based on the log type, the target abnormal synchronization task log and the log index are stored in a default database's abnormal list.

[0021] Optionally, the step of indexing the category list of the preset database according to the retrieval request to obtain retrieval results includes:

[0022] The search request is parsed to obtain preliminary search criteria;

[0023] The initial search conditions are initialized to obtain the initial query parameters;

[0024] The initialization query parameters are encapsulated into an SQL query statement;

[0025] Based on the SQL query statement, the index is used to query the corresponding category list in the preset database to obtain the first search result;

[0026] When a page-turning instruction is received, a database snapshot with since_id as the primary key is generated based on the first log ID of the first search result;

[0027] Based on the database snapshot and the SQL query statement, the corresponding category list in the preset database is queried to obtain a preset number of page-turning search results.

[0028] Optionally, the step of querying the corresponding category list in the preset database based on the database snapshot and the SQL query statement to obtain a preset number of pagination search results includes:

[0029] Based on the database snapshot, confirm the query range of the corresponding category list in the preset database;

[0030] The SQL query statement is used to perform an index query within the query range to obtain a preset number of paginated retrieval results.

[0031] Optionally, after performing an index query on the category list of the preset database according to the retrieval request to obtain the retrieval results, the method further includes:

[0032] Count the number of abnormal synchronization task logs for each log type;

[0033] Based on the log types and their corresponding quantities, construct an abnormal log risk warning chart.

[0034] Optionally, after storing the synchronization task log and the log index in a preset database category list according to the synchronization task log object, the method further includes:

[0035] Delete the synchronization task logs and log indexes in the preset database that have exceeded the preset storage time at the preset time point.

[0036] To address the aforementioned problems, the present invention also provides a device for collecting and visualizing synchronous task logs, the device comprising:

[0037] The log object creation module is used to create a synchronization task log object corresponding to the synchronization task in response to the start of the synchronization task;

[0038] The index creation module is used to create a log index corresponding to the synchronization task logs generated by the synchronization task based on the synchronization task object and the source system of the synchronization task.

[0039] The log storage module is used to store the synchronization task log and the log index in a category list of a preset database according to the synchronization task log object;

[0040] The log retrieval module is used to respond to a received retrieval request, perform an index query on the category list of the preset database according to the retrieval request, and obtain retrieval results.

[0041] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0042] At least one processor; and,

[0043] A memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the synchronous task log collection and visualization retrieval method described above.

[0045] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the aforementioned synchronous task log collection and visualization retrieval method.

[0046] This invention creates a synchronization task corresponding to the synchronization task in response to the start of the synchronization task.

[0047] Based on the synchronization task object and the source system of the synchronization task, a log index corresponding to the synchronization task log generated by the synchronization task is created. According to the synchronization task log object, the synchronization task log and the log index are stored in a category list of a preset database. One synchronization task corresponds to one synchronization task log object, ensuring the standardization and uniqueness of the logs, which helps improve the accuracy of log collection. In response to a received retrieval request, the system performs the following steps based on the retrieval request:

[0048] The pre-defined database category list is used for indexing and querying to obtain retrieval results. The corresponding synchronization task log can be quickly found based on the log index, thereby improving retrieval speed. Therefore, the synchronization task proposed in this invention...

[0049] The method, device, electronic equipment, and computer-readable storage medium for log collection and visualization retrieval can solve the problems of low accuracy in log collection and slow retrieval speed. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a method for collecting and visually retrieving synchronous task logs according to an embodiment of the present invention.

[0051] Figure 2 for Figure 1 The diagram shows a detailed implementation flow of one step in the synchronous task log collection and visualization retrieval method.

[0052] Figure 3 for Figure 1 The diagram shows a detailed implementation flow of another step in the synchronous task log collection and visualization retrieval method.

[0053] Figure 4A functional block diagram of a synchronous task log collection and visualization retrieval device provided in an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the synchronous task log collection and visualization retrieval method according to an embodiment of the present invention.

[0055] 5. The realization of the objective of the present invention, its functional characteristics and advantages will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0056] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0057] This application provides a method for collecting and visualizing synchronous task logs. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0058] Reference Figure 1 The diagram shown is a flowchart illustrating a method for collecting and visually retrieving synchronization task logs according to an embodiment of the present invention. In this embodiment, the method for collecting and visually retrieving synchronization task logs includes:

[0059] S1. In response to the start of the synchronization task, create a synchronization task log object corresponding to the synchronization task.

[0060] In one embodiment of the present invention, in response to the start of each synchronization task, a new synchronization task log object is immediately created. The synchronization task log object is instantiated according to log initialization parameters and provides an add log function to save the log in a preset database. For example, a new synchronization task log object synclog is created, and the synchronization task log object synclog can provide an add log function addSyncLog to save the log in the preset database.

[0061] In this embodiment of the invention, one synchronization task corresponds to one synchronization task log object, ensuring the standardization and uniqueness of the logs, which helps to improve the accuracy of log collection.

[0062] S2. Based on the synchronization task object and source system of the synchronization task, create a log index corresponding to the synchronization task log generated by the synchronization task.

[0063] In one embodiment of the present invention, the synchronization task object and source system of the synchronization task log are added as log indexes to a preset position in the synchronization task log generated by the synchronization task. The synchronization task object corresponds one-to-one with the synchronization task. For example, the synchronization task object SyncObject and source system SourceSystem of the synchronization task log can be added as log indexes to a preset position in the synchronization task log generated by the synchronization task. Using the index for data querying can speed up data retrieval and strengthen the connections between tables in the database.

[0064] S3. Based on the synchronization task log object, save the synchronization task log and the log index in a category list of a preset database.

[0065] For details, please refer to Figure 2 As shown, S3 includes:

[0066] S31. Initialize log parameters to obtain log initialization parameters;

[0067] S32. Instantiate the synchronization task log object according to the log initialization parameters to obtain the target synchronization task log object;

[0068] S33. Determine whether the synchronization task log is abnormal;

[0069] When the synchronization task log is abnormal, S34, according to the add log function in the target synchronization task log object, the abnormal synchronization task log and the log index are saved in the abnormal list of the preset database.

[0070] When the synchronization task log is normal, S35, according to the add log function in the target synchronization task log object, save the normal synchronization task log and the log index in the normal list of the preset database.

[0071] S36. Clear the target synchronization task log object.

[0072] In one embodiment of the present invention, the log initialization parameter is a parameter that does not change with the consumption of the synchronization task, such as the source system parameter.

[0073] In one embodiment of the present invention, in response to the start of a synchronization task, a new synchronization task log object `synclog` is immediately created, and log parameters are initialized to obtain log initialization parameters. The synchronization task log object is then instantiated based on these log initialization parameters to obtain a target synchronization task log object. When the synchronization task ends, a synchronization task log is generated. It is determined whether the synchronization task log is abnormal. The existence of the `addSyncLog` function in the target synchronization task log object is checked. If the `addSyncLog` function does not exist, the process returns to creating a new synchronization task log object `synclog`, and the synchronization task log object corresponding to the synchronization task is reconstructed. When the target synchronization task log object exists, abnormal synchronization task logs are saved in the abnormal list of a preset database according to their `log_id` order, and normal synchronization task logs are saved in the normal list of a preset database according to their `log_id` order. After a synchronization task log is successfully added, the synchronization task log object and the target synchronization task log object are destroyed. When a new synchronization task occurs, a corresponding synchronization task log object is reconstructed.

[0074] In this embodiment of the invention, each time a synchronization task log is added, it is determined whether the addSyncLog function exists in the corresponding synchronization task log object, and after the synchronization task log is successfully added, the standardization and uniqueness of each synchronization task log are guaranteed.

[0075] Further, S34 includes:

[0076] Parse the abnormal synchronization task log to obtain the target abnormal synchronization task log in a preset format;

[0077] The target abnormal synchronization task logs are classified according to application scenarios to obtain log types;

[0078] Based on the log type, the target abnormal synchronization task log and the log index are stored in a default database's abnormal list.

[0079] In this embodiment of the invention, the method for saving the normal synchronization task log is the same as the method for saving the abnormal synchronization task log, and will not be described in detail here.

[0080] In this embodiment of the invention, the preset database can be a MySQL database.

[0081] In one embodiment of the present invention, the abnormal synchronization task log can be parsed into a target abnormal synchronization task log including parameters such as synchronization task object SyncObject, source system SourceSystem, log status Status, creation time Createtime, duration Duration, and log description Desc.

[0082] In this embodiment of the invention, the synchronization tasks come from multiple systems, and the application scenarios corresponding to the logs are also different. In order to ensure the speed of the log retrieval interface, it is necessary to classify and store the logs according to the application scenarios to improve the query speed.

[0083] In one embodiment of the present invention, the BERT model can be used to classify the abnormal synchronization task logs and the normal synchronization task logs according to their application scenarios.

[0084] In this embodiment of the invention, after step S3, the method further includes:

[0085] Delete the synchronization task logs and log indexes in the preset database that have exceeded the preset storage time at the preset time point.

[0086] Since the synchronization task logs are generated frequently and have a large cumulative number, in this embodiment of the invention, the synchronization task logs and log indexes older than one month can be deleted from the preset database at midnight every day to reduce disk pressure and shorten the interface query speed.

[0087] In this embodiment of the invention, the synchronization task logs are categorized and stored in the preset database, which can reduce the pressure on the database and improve the speed of subsequent retrieval and query.

[0088] S4. In response to the received search request, perform an index query on the category list of the preset database according to the search request to obtain the search results.

[0089] In one embodiment of the present invention, the retrieval request may include log filtering conditions input by the user according to the log filtering condition selector of the preset visualization page, and retrieval conditions input by the user; wherein the log filtering conditions include parameters such as: SyncObject, SourceSystem, Log Status, Creation Time, Duration, and PageSize, etc.

[0090] For details, please refer to Figure 3 As shown, step S4, which involves indexing the category list of the preset database according to the retrieval request to obtain retrieval results, includes:

[0091] S41. Parse the search request to obtain preliminary search conditions;

[0092] S42. Initialize the preliminary search conditions to obtain the initial query parameters;

[0093] S43. Encapsulate the initialization query parameters into an SQL query statement;

[0094] S44. Based on the SQL query statement, query the corresponding category list in the preset database to obtain the first search result;

[0095] S45. When a page-turning instruction is received, a database snapshot with since_id as the primary key is generated based on the first log ID of the first search result.

[0096] S46. Based on the database snapshot and the SQL query statement, query the corresponding category list in the preset database to obtain a preset number of page-turning search results.

[0097] In one embodiment of the present invention, a NodeJS program combined with an MVC architecture can be used to provide a visual log query interface, so that different business interfaces can reuse the same SQL statement in the database. For example, the interface for querying abnormal synchronization task logs and normal synchronization task logs reuses a function of the Model layer.

[0098] Further, S46 includes:

[0099] Based on the database snapshot, confirm the query range of the corresponding category list in the preset database;

[0100] The SQL query statement is used to perform an index query within the query range to obtain a preset number of paginated retrieval results.

[0101] In one embodiment of the present invention, the first search result is a preset number of search results. For example, the search results displayed on the first screen are 50 synchronization task logs. When the user performs a page-turning action, the log_id of the first synchronization task log in the 50 synchronization task logs generates a database snapshot with since_id as the primary key. Furthermore, the query range in the preset database is determined according to the since_id corresponding to the database snapshot, avoiding repeated queries in the already queried data and improving retrieval efficiency.

[0102] In one embodiment of the present invention, during data retrieval, the log visualization page can also provide a record of the user's last selected query parameters. This parameter record is stored in the front-end indexdb, allowing users to easily obtain high-frequency synchronization task logs in real time by refreshing the page. Typically, the number of synchronization task logs involved in the retrieval is large, requiring pagination. Traditional pagination queries only retrieve data for each page based on time, pagesize, and pagenumber. However, logs are generated very frequently, and multiple logs may be generated at the same time, potentially leading to duplicate data during pagination queries. However, by forming a database snapshot for each page based on the log ID of the synchronization task logs, the sorting stability of the retrieved high-refresh data is ensured, which helps improve retrieval efficiency.

[0103] In this embodiment of the invention, log query filtering can be performed based on the index established by the synchronization task log object and the source system of the synchronization task. During the retrieval query, the corresponding synchronization task log can be quickly found based on the index, thereby improving the retrieval speed.

[0104] Furthermore, after performing an index query on the category list of the preset database according to the retrieval request to obtain the retrieval results, the method further includes:

[0105] Count the number of abnormal synchronization task logs for each log type;

[0106] Based on the log types and their corresponding quantities, construct an abnormal log risk warning chart.

[0107] In this embodiment of the invention, when the number of abnormal synchronization task logs of the same log type exceeds a preset threshold, the corresponding abnormal synchronization task logs can be pushed to preset technical or management personnel for abnormal synchronization task log analysis.

[0108] This invention, in response to the start of a synchronization task, creates a synchronization task log object corresponding to the synchronization task. Based on the synchronization task object and the source system, it creates a log index corresponding to the synchronization task logs generated by the synchronization task. According to the synchronization task log object, the synchronization task logs and the log index are stored in a category list of a preset database. One synchronization task corresponds to one synchronization task log object, ensuring the standardization and uniqueness of the logs, which helps improve the accuracy of log collection. In response to a received retrieval request, it performs an index query on the category list of the preset database according to the retrieval request to obtain the retrieval results. The corresponding synchronization task log can be quickly found based on the log index, thereby improving retrieval speed. Therefore, the synchronization task log collection and visualization retrieval method proposed in this invention can solve the problems of low log collection accuracy and slow retrieval speed.

[0109] like Figure 4 The diagram shown is a functional block diagram of a synchronous task log collection and visualization retrieval device provided in an embodiment of the present invention.

[0110] The synchronous task log collection and visualization retrieval device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the synchronous task log collection and visualization retrieval device 100 may include a log object creation module 101, an index creation module 102, a log storage module 103, and a log retrieval module. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0111] In this embodiment, the functions of each module / unit are as follows:

[0112] The log object creation module 101 is used to create a synchronization task log object corresponding to the synchronization task in response to the start of the synchronization task.

[0113] The index creation module 102 is used to create a log index corresponding to the synchronization task log generated by the synchronization task based on the synchronization task object and the source system of the synchronization task.

[0114] The log storage module 103 is used to store the synchronization task log and the log index in a category list of a preset database according to the synchronization task log object.

[0115] The log retrieval module 104 is used to respond to a received retrieval request, and to perform an index query on the category list of the preset database according to the retrieval request to obtain retrieval results.

[0116] In detail, the modules in the synchronous task log collection and visualization retrieval device 100 described in this embodiment of the invention employ the same methods as described above. Figures 1 to 3 The method used is the same as the synchronous task log collection and visualization retrieval method described in the previous section, and can produce the same technical effect, so it will not be repeated here.

[0117] like Figure 5 The diagram shown is a structural schematic of an electronic device for implementing a synchronous task log collection and visualization retrieval method according to an embodiment of the present invention.

[0118] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a synchronous task log collection and visualization retrieval program.

[0119] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing synchronous task log collection and visualization retrieval programs) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0120] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code for synchronous task log collection and visualization retrieval programs, but also to temporarily store data that has been output or will be output.

[0121] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0122] The communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0123] Figure 5 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 5 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0124] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0125] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0126] The synchronous task log collection and visualization retrieval program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0127] In response to the start of a synchronization task, a synchronization task log object corresponding to the synchronization task is created;

[0128] Based on the synchronization task object and source system of the synchronization task, create a log index corresponding to the synchronization task log generated by the synchronization task;

[0129] Based on the synchronization task log object, the synchronization task log and the log index are stored in a category list of a preset database;

[0130] In response to a received search request, the system performs an index query on the category list of the preset database according to the search request to obtain search results.

[0131] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.

[0132] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0133] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0134] In response to the start of a synchronization task, a synchronization task log object corresponding to the synchronization task is created;

[0135] Based on the synchronization task object and source system of the synchronization task, create a log index corresponding to the synchronization task log generated by the synchronization task;

[0136] Based on the synchronization task log object, the synchronization task log and the log index are stored in a category list of a preset database;

[0137] In response to a received search request, the system performs an index query on the category list of the preset database according to the search request to obtain search results.

[0138] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0139] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0140] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0141] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0142] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0143] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0144] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0145] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for collecting and visually retrieving synchronous task logs, characterized in that, The method includes: In response to the start of a synchronization task, a synchronization task log object corresponding to the synchronization task is created; The synchronization task object and source system of the synchronization task log are used as log indexes, and the log indexes are added to a preset position in the synchronization task log generated by the synchronization task. Initialize the log parameters to obtain the log initialization parameters, and instantiate the synchronization task log object according to the log initialization parameters to obtain the target synchronization task log object; Determine whether the synchronization task log is abnormal. When the synchronization task log is abnormal, parse the abnormal synchronization task log to obtain a target abnormal synchronization task log in a preset format. Classify the target abnormal synchronization task log by application scenario to obtain the log type. According to the log type, store the target abnormal synchronization task log and the log index in a preset database in an abnormal list. When the synchronization task log is normal, the normal synchronization task log and the log index are saved in the normal list of the preset database according to the add log function in the target synchronization task log object. In response to a received search request, the search request is parsed, initialized, and parameters are encapsulated to obtain an SQL query statement. Based on the SQL query statement in the search request, an index query is performed on the category list of the preset database to obtain search results.

2. The method of claim 1, wherein, The search request is parsed, initialized, and parameterized to obtain an SQL query statement. Based on the SQL query statement, an index query is performed on the category list of the preset database to obtain search results, including: The search request is parsed to obtain preliminary search criteria; The initial search conditions are initialized to obtain the initial query parameters; The initialization query parameters are encapsulated into an SQL query statement; Based on the SQL query statement, the index is used to query the corresponding category list in the preset database to obtain the first search result; When a page-turning instruction is received, a database snapshot with since_id as the primary key is generated based on the first log ID of the first search result; Based on the database snapshot and the SQL query statement, the corresponding category list in the preset database is queried to obtain a preset number of page-turning search results.

3. The method of claim 2, wherein the method further comprises: The step of querying the corresponding category list in the preset database based on the database snapshot and the SQL query statement to obtain a preset number of pagination search results includes: Based on the database snapshot, confirm the query range of the corresponding category list in the preset database; The SQL query statement is used to perform an index query within the query range to obtain a preset number of paginated retrieval results.

4. The method of claim 1, wherein the method further comprises: After performing an index query on the category list of the preset database according to the retrieval request to obtain the retrieval results, the method further includes: Count the number of abnormal synchronization task logs for each log type; Based on the log types and their corresponding quantities, construct an abnormal log risk warning chart.

5. The method for collecting and visually retrieving synchronous task logs as described in claim 1, characterized in that, After storing the synchronization task log and the log index in a preset database category list according to the synchronization task log object, the method further includes: Delete the synchronization task logs and log indexes in the preset database that have exceeded the preset storage time at the preset time point.

6. A device for collecting and visualizing synchronous task logs, characterized in that, The device includes: The log object creation module is used to create a synchronization task log object corresponding to the synchronization task in response to the start of the synchronization task; The index creation module is used to use the synchronization task object and source system of the synchronization task log as a log index, and add the log index to a preset position in the synchronization task log generated by the synchronization task. The log storage module is used to initialize log parameters to obtain log initialization parameters, and instantiate the synchronization task log object according to the log initialization parameters to obtain the target synchronization task log object; The log storage module is also used to determine whether the synchronization task log is abnormal. When the synchronization task log is abnormal, the abnormal synchronization task log is parsed to obtain a target abnormal synchronization task log in a preset format. The target abnormal synchronization task log is classified by application scenario to obtain the log type. According to the log type, the target abnormal synchronization task log and the log index are stored in a preset database in an abnormal list. The log storage module is also used to save the normal synchronization task log and the log index in the normal list of the preset database according to the add log function in the target synchronization task log object when the synchronization task log is normal. The log retrieval module is used to respond to a received retrieval request by parsing, initializing, and encapsulating parameters of the retrieval request to obtain an SQL query statement, and then performing an index query on the category list of the preset database according to the SQL query statement in the retrieval request to obtain the retrieval results.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the synchronous task log collection and visualization retrieval method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the synchronous task log collection and visualization retrieval method as described in any one of claims 1 to 5.

Images