Data processing method and device, computer equipment and storage medium

Abstract

The invention belongs to the technical field of data processing, and relates to a data processing method and device, computer equipment and a storage medium, and the method comprises the steps: adding a specified tag to a request header of a request based on an AOP tangent surface in a target project before a client sends the request; intercepting the request based on a processing component in the target project, recording a specified label corresponding to the request, and recording starting time and ending time of the request; constructing a corresponding tracking information object based on the specified label, the starting time and the ending time; receiving a tracking information object based on a collector, and obtaining specified request data corresponding to a specified label contained in the tracking information object; obtaining link tracking data corresponding to the request based on assembly of the tracking information object and the specified request data; and storing the link tracking data in a database. In addition, the link tracking data can be stored in the block chain. According to the invention, the processing workload of the positioning problem is reduced, and the processing efficiency of positioning problem processing is improved.

Description

Technical Field

[0001] The present application relates to the fields of data processing technology and financial technology, and in particular to data processing methods, devices, computer equipment and storage media. Background Art

[0002] In the software development and testing process in the financial field, the traditional problem handling mechanism faces many challenges, especially in the problem diagnosis and repair process in the testing phase and production environment, where the inefficiency of this mechanism is particularly significant. Specifically, when developers complete the development of a function or module, they will submit it to the testing team for testing and verification. During this process, testers conduct comprehensive and detailed testing of the software based on established test cases and scenarios. However, once defects or anomalies are found during the testing process or after the software is deployed to the production environment, testers often need to spend a lot of time and energy to locate and fix the problem.

[0003] However, the process of locating the problem itself is a tedious task that may involve multiple steps and multiple environment configurations. It requires testers to accurately record the conditions, operation steps, and related system status information when the problem occurs. After completing the reproduction, testers can usually only feedback the problem and its reproduction steps to the development team in the form of a report, and then enter a long waiting period, that is, waiting for the developers to locate the root cause of the problem, develop a repair plan, and implement the repair. During this process, developers may need to repeatedly ask testers for more details, or need to reproduce the problem in different development and test environments, which further prolongs the time to solve the problem.

[0004] Therefore, the problem handling mechanism used in the existing software development and testing process in the financial field has the problems of heavy workload and low processing efficiency in the process of locating problems. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a data processing method, device, computer equipment and storage medium to solve the technical problems of large workload and low processing efficiency in the process of locating problems in the problem handling mechanism used in the existing software development and testing process in the financial field.

[0006] In order to solve the above technical problems, the present application provides a data processing method, which adopts the following technical solution:

[0007] Before the client sends a request, add a specified tag to the request header based on the AOP aspect in the target project that was previously introduced;

[0008] intercepting the request based on the processing component in the target project, recording the specified tag corresponding to the request, and recording the start time and end time of the request;

[0009] Constructing a corresponding tracking information object based on the specified tag, the start time and the end time;

[0010] Receive the tracking information object based on a preset collector, and obtain specified request data corresponding to the specified tag included in the tracking information object;

[0011] Assembling the tracking information object and the specified request data to obtain link tracking data corresponding to the request;

[0012] The link tracking data is stored in a preset database.

[0013] Furthermore, the step of receiving the tracking information object based on a preset collector and obtaining the specified request data corresponding to the specified tag included in the tracking information object specifically includes:

[0014] Performing data verification on the tracking information object based on a preset verification strategy;

[0015] If the tracking information object passes the data verification, extracting the specified tag from the tracking information object;

[0016] Call the preset storage structure;

[0017] Searching for other request data related to the specified tag from the storage structure;

[0018] The other request data is used as the designated request data.

[0019] Furthermore, the step of assembling the tracking information object and the specified request data to obtain the link tracking data corresponding to the request specifically includes:

[0020] Get the preset sorting strategy;

[0021] Assembling the tracking information object and the specified request data based on the sorting strategy to obtain a complete link;

[0022] Encapsulating the link to obtain a corresponding link tracking object;

[0023] The link tracking object is used as the link tracking data.

[0024] Furthermore, after the step of receiving the tracking information object based on the preset collector and obtaining the specified request data corresponding to the specified tag included in the tracking information object, the method further includes:

[0025] Call the preset asynchronous processing aspect;

[0026] Get the preset alarm rules;

[0027] By means of the asynchronous processing aspect, judging whether the tracking information object meets the alarm condition according to the alarm rule;

[0028] If so, generating corresponding warning information based on the tracking information object;

[0029] Based on the preset message push method, the alarm information is pushed to relevant personnel.

[0030] Furthermore, before the step of obtaining the preset alarm rule, the method further includes:

[0031] Determine whether a user-triggered alarm rule generation request is received;

[0032] If yes, display the preset rule management interface;

[0033] Receiving rule configuration information input by the user in the rule management interface;

[0034] Constructing corresponding alarm rules based on the rule configuration information;

[0035] The alarm rules are stored in the database.

[0036] Further, after the step of storing the link tracking data in a preset database, the method further includes:

[0037] Performing data analysis on the link tracking data to obtain corresponding analysis data;

[0038] Call the preset first display interface;

[0039] The link tracking data and the analysis data are displayed and processed based on the first display interface.

[0040] Further, after the step of storing the link tracking data in a preset database, the method further includes:

[0041] Acquire the collected specified data from the database based on a preset timed task;

[0042] Call the preset abnormal analysis model;

[0043] Performing an abnormality analysis on the specified data based on the abnormality analysis model to obtain corresponding abnormality analysis results;

[0044] Obtaining a specified solution strategy corresponding to the abnormal analysis result;

[0045] The abnormality analysis result and the specified solution strategy are displayed based on a preset second display interface.

[0046] In order to solve the above technical problems, the embodiment of the present application further provides a data processing device, which adopts the following technical solution:

[0047] Add a module to add a specified tag to the request header of the request based on the AOP aspect in the pre-introduced target project before the client sends a request;

[0048] A recording module, configured to intercept the request based on the processing component in the target project, record the specified tag corresponding to the request, and record the start time and end time of the request;

[0049] A first construction module, configured to construct a corresponding tracking information object based on the specified tag, the start time and the end time;

[0050] A first acquisition module, configured to receive the tracking information object based on a preset collector, and acquire specified request data corresponding to a specified tag included in the tracking information object;

[0051] An assembling module, configured to assemble the link tracking data corresponding to the request based on the tracking information object and the specified request data;

[0052] The first storage module is used to store the link tracking data in a preset database.

[0053] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0054] Before the client sends a request, add a specified tag to the request header based on the AOP aspect in the target project that was previously introduced;

[0055] intercepting the request based on the processing component in the target project, recording the specified tag corresponding to the request, and recording the start time and end time of the request;

[0056] Constructing a corresponding tracking information object based on the specified tag, the start time and the end time;

[0057] Receive the tracking information object based on a preset collector, and obtain specified request data corresponding to the specified tag included in the tracking information object;

[0058] Assembling the tracking information object and the specified request data to obtain link tracking data corresponding to the request;

[0059] The link tracking data is stored in a preset database.

[0060] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0061] Before the client sends a request, add a specified tag to the request header based on the AOP aspect in the target project that was previously introduced;

[0062] intercepting the request based on the processing component in the target project, recording the specified tag corresponding to the request, and recording the start time and end time of the request;

[0063] Constructing a corresponding tracking information object based on the specified tag, the start time and the end time;

[0064] Receive the tracking information object based on a preset collector, and obtain specified request data corresponding to the specified tag included in the tracking information object;

[0065] Assembling the tracking information object and the specified request data to obtain link tracking data corresponding to the request;

[0066] The link tracking data is stored in a preset database.

[0067] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0068] Before the client sends a request, the application first adds a specified tag to the request header of the request based on the AOP aspect in the pre-introduced target project; then intercepts the request based on the processing component in the target project, records the specified tag corresponding to the request, and records the start time and end time of the request; then builds a corresponding tracking information object based on the specified tag, the start time and the end time; subsequently receives the tracking information object based on a preset collector, and obtains the specified request data corresponding to the specified tag contained in the tracking information object; further assembles the link tracking data corresponding to the request based on the tracking information object and the specified request data; finally, stores the link tracking data in a preset database. Before the client sends a request, the application adds a specified tag to the request header of the request based on the AOP aspect in the target project introduced in advance, then intercepts the request based on the processing component in the target project, and builds a corresponding tracking information object based on the recorded specified tag, start time and end time corresponding to the request. After that, the tracking information object is received based on the collector, and the specified request data corresponding to the specified tag contained in the tracking information object is obtained, and then the link tracking data corresponding to the request is assembled based on the tracking information object and the specified request data. Finally, the link tracking data is stored in a preset database, so that the link tracking function of the log can be quickly and accurately implemented based on the use of the database in the future, thereby helping developers and testers to quickly locate problems, effectively reducing the processing workload of locating problems, and improving the processing efficiency of locating problems, which is conducive to improving work efficiency and work quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0070] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0071] Figure 2 A flowchart of an embodiment of a data processing method according to the present application;

[0072] Figure 3 is a schematic structural diagram of an embodiment of a data processing device according to the present application;

[0073] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0075] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0076] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0077] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0078] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0079] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0080] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0081] It should be noted that the data processing method provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the data processing device is generally arranged in the server / terminal device.

[0082] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0083] Continue to refer Figure 2 , shows a flow chart of an embodiment of a data processing method according to the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The data processing method provided in the embodiment of the present application can be applied to any scenario where data processing is required, and the data processing method can be applied to products in these scenarios, for example, data processing in the financial insurance field. The data processing method comprises the following steps:

[0084] Step S201, before the client sends a request, adding a specified tag to the request header of the request based on the AOP aspect in the pre-introduced target project.

[0085] In this embodiment, the electronic device (eg, Figure 1The server / terminal device shown in the figure) can obtain the request through a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future. The execution subject of this application may be a log link tracking system, or simply referred to as a system. The architecture of the system includes: a client, a collector (Collector), and may also include an interface. The above-mentioned client may be an application of a developer. A target project Spring Starter is introduced in advance. The target project includes an AOP aspect and a Filter processing component. Before the client sends a request and enters the Spring container, a unique label (specified label) is added to the request header through the AOP aspect. In addition, the request is intercepted by the Filter processing component, and a complete tracking link is assembled according to the unique label, and then the data is sent to the collector in an asynchronous manner. The collector is used to receive and process data from the client and persist the data to a database. The above page provides visual link tracking and data analysis functions. In the financial insurance business scenario, the above processing requests may include transaction requests, payment requests, claims requests, etc. In addition, the log link tracking system supports log tracking of distributed microservices and cloud architecture services, and supports third-party service log link tracking, non-intrusive. In addition, non-technical personnel can also easily locate the business link where the problem occurs through the UI interface.

[0086] Step S202: intercepting the request based on the processing component in the target project, recording the designated tag corresponding to the request, and recording the start time and end time of the request.

[0087] In this embodiment, the processing component is a Filter processing component. The Filter processing component can be used to intercept the request, record the specified tag corresponding to the request, and simultaneously record the start time and end time of the request.

[0088] Step S203: construct a corresponding tracking information object based on the specified tag, the start time and the end time.

[0089] In this embodiment, the obtained designated tag, start time and end time can be encapsulated to obtain the corresponding tracking information object. In addition, the recorded data, namely the tracking information object, can be sent to the sending collector through the message middleware.

[0090] Step S204: receiving the tracking information object based on a preset collector, and obtaining specified request data corresponding to the specified tag included in the tracking information object.

[0091] In this embodiment, the specific implementation process of obtaining the specified request data corresponding to the specified tag included in the tracking information object will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0092] Step S205: assembling the link tracking data corresponding to the request based on the tracking information object and the specified request data.

[0093] In this embodiment, the specific implementation process of assembling the tracking information object and the specified request data to obtain the link tracking data corresponding to the request will be described in further detail in subsequent specific embodiments of the present application and will not be elaborated on here.

[0094] Step S206, storing the link tracking data in a preset database.

[0095] In this embodiment, the link tracking data can be consumed by using a collector and persisting the link tracking data to a database, so that complete link tracking can be provided. Among them, for developers, during the development process or troubleshooting process, they can give priority to viewing the link tracking of the log link tracking system, quickly locate a certain link, and directly extract the log of the link from the database according to the log identifier, so that interface development and joint debugging, including problem handling, can be completed more efficiently. In addition, the system can also provide an intuitive web visualization interface to clearly track the abnormality of a specific link, and can also count the visual data such as the abnormality rate.

[0096] Before the client sends a request, the application first adds a specified tag to the request header of the request based on the AOP aspect in the pre-introduced target project; then intercepts the request based on the processing component in the target project, records the specified tag corresponding to the request, and records the start time and end time of the request; then builds a corresponding tracking information object based on the specified tag, the start time and the end time; subsequently receives the tracking information object based on a preset collector, and obtains the specified request data corresponding to the specified tag contained in the tracking information object; further assembles the link tracking data corresponding to the request based on the tracking information object and the specified request data; finally, stores the link tracking data in a preset database. Before the client sends a request, the application adds a specified tag to the request header of the request based on the AOP aspect in the target project introduced in advance, then intercepts the request based on the processing component in the target project, and builds a corresponding tracking information object based on the recorded specified tag, start time and end time corresponding to the request. After that, the tracking information object is received based on the collector, and the specified request data corresponding to the specified tag contained in the tracking information object is obtained, and then the link tracking data corresponding to the request is assembled based on the tracking information object and the specified request data. Finally, the link tracking data is stored in a preset database, so that the link tracking function of the log can be quickly and accurately implemented based on the use of the database in the future, thereby helping developers and testers to quickly locate problems, effectively reducing the processing workload of locating problems, and improving the processing efficiency of locating problems, which is conducive to improving work efficiency and work quality.

[0097] In some optional implementations, step S204 includes the following steps:

[0098] The tracking information object is subjected to data verification based on a preset verification strategy.

[0099] In this embodiment, the policy content of the above verification policy may include: checking whether all required fields have been filled in, whether the value of the field meets expectations (for example, whether the user ID is a positive integer, whether the timestamp is a valid date and time format, etc.). Among them, the tracking information object can be data verified according to the policy content of the verification policy, so as to obtain the corresponding data verification result. In addition, if the data verification result is that the data verification fails, an error response will be returned to the front end, indicating that the sent data is incorrect.

[0100] If the tracking information object passes the data verification, the specified tag is extracted from the tracking information object.

[0101] In this embodiment, the required designated tag can be obtained by parsing the above tracking information object.

[0102] Recall the preset storage structure.

[0103] In this embodiment, the storage structure may be a Map in memory, a cache (such as Redis), or another database. The data stored in the storage structure may include the user's previous browsing history, the record of adding to the shopping cart, the submitted order record, etc.

[0104] Other request data related to the specified tag is searched from the storage structure.

[0105] In this embodiment, data query may be performed on the storage structure according to the above-mentioned designated tag to find out other request data related to the designated tag.

[0106] The other request data is used as the designated request data.

[0107] The present application performs data verification on the tracking information object based on a preset verification strategy; if the tracking information object passes the data verification, the specified tag is extracted from the tracking information object; then the preset storage structure is called; thereafter, other request data related to the specified tag is found from the storage structure; and the other request data is subsequently used as the specified request data. After receiving the tracking information object, the present application will also intelligently perform data verification on the tracking information object based on the use of the verification strategy to ensure the data accuracy and compliance of the tracking information object. When the tracking information object passes the data verification, the specified tag is extracted from the tracking information object, and then other request data related to the specified tag is found from the storage structure, thereby achieving rapid and accurate acquisition of the specified request data corresponding to the specified tag, improving the efficiency of acquiring the specified request data, and ensuring the data accuracy of the obtained specified request data.

[0108] In some optional implementations of this embodiment, step S205 includes the following steps:

[0109] Get the preset sorting strategy.

[0110] In this embodiment, the above-mentioned sorting strategy may adopt a time strategy, or may also adopt other logics, such as operation type.

[0111] The tracking information object and the specified request data are assembled based on the sorting strategy to obtain a complete link.

[0112] In this embodiment, after obtaining the specified request data according to the specified tag, the tracking information object and the specified request data can be further assembled according to the sorting strategy, which may include integration and processing to generate a more structured, easy to understand and analyze complete tracking link, that is, the above link. Exemplarily, the above link is the complete process of the specified tag flow, for example, a link that can clearly show the entire process from the user browsing the product to the final payment.

[0113] The link is encapsulated to obtain a corresponding link tracking object.

[0114] In this embodiment, the assembled link is encapsulated into a new object, namely, a link tracking object, which may include the user ID, the link ID, and each node in the link (each node includes information such as the request time, type, and product ID).

[0115] The link tracking object is used as the link tracking data.

[0116] The present application obtains a preset sorting strategy; then assembles the tracking information object and the specified request data based on the sorting strategy to obtain a complete link; then encapsulates the link to obtain a corresponding link tracking object; and subsequently uses the link tracking object as the link tracking data. The present application assembles the tracking information object and the specified request data based on the use of a sorting strategy to obtain a complete link, and then encapsulates the link, so that the corresponding link tracking data can be obtained quickly and accurately, thereby improving the generation efficiency of the link tracking data and ensuring the data accuracy of the obtained link tracking data.

[0117] In some optional implementations, after step S205, the electronic device may further perform the following steps:

[0118] Call the preset asynchronous processing aspect.

[0119] In this embodiment, an asynchronous processing aspect is added to the collector in advance to implement alarm rule judgment. Specifically, an AOP aspect class is created in the collector. An entry point is defined for the received tracking information object. And in the interception method, it is determined whether the alarm information needs to be sent according to the alarm rule.

[0120] Get the preset alarm rules.

[0121] In this embodiment, alarm rules may be predefined according to actual needs of users, such as response time exceeding a threshold, error rate exceeding a threshold, etc. Then, in the interception method, it is determined whether an alarm needs to be triggered according to the alarm rules.

[0122] Through the asynchronous processing aspect, it is determined according to the alarm rule whether the tracking information object meets the alarm condition.

[0123] In this embodiment, by using the asynchronous processing aspect, it is determined whether the tracking information object needs to trigger an alarm according to the rule content of the alarm rule. If the alarm needs to be triggered, it is determined that the tracking information object meets the alarm conditions; if the alarm does not need to be triggered, it is determined that the tracking information object does not meet the alarm conditions.

[0124] If so, generate corresponding warning information based on the tracking information object.

[0125] In this embodiment, an alarm information template matching the alarm rule is pre-constructed according to the actual alarm demand, and then the above-mentioned tracking information object is filled into the corresponding position in the alarm information, so as to generate the corresponding alarm information.

[0126] Based on the preset message push method, the alarm information is pushed to relevant personnel.

[0127] In this embodiment, there is no specific limitation on the selection of the message push method, and for example, it may include SMS, email, interface display, etc. The above-mentioned relevant personnel may be the personnel responsible for operation and maintenance.

[0128] This application calls the preset asynchronous processing aspect; then obtains the preset alarm rules; then, through the asynchronous processing aspect, according to the alarm rules, determines whether the tracking information object meets the alarm conditions; if so, generates corresponding alarm information based on the tracking information object; and then pushes the alarm information to relevant personnel based on the preset message push method. After receiving the tracking information object through the collector, this application will also intelligently perform alarm detection on the tracking information object based on the use of asynchronous processing aspects and alarm rules, thereby improving the automaticity, intelligence and accuracy of alarm detection. In addition, when it is detected that the tracking information object meets the alarm conditions, the corresponding alarm information will be automatically generated based on the tracking information object, and the alarm information will be pushed to relevant personnel based on the preset message push method, thereby realizing automatic alarm processing of abnormal tracking information objects, which is conducive to the relevant users to be able to view the alarm information and timely understand the current abnormalities, and then be able to take corresponding processing measures, which is conducive to improving the abnormality handling efficiency of relevant personnel and improving the work experience of relevant personnel.

[0129] In some optional implementations, before the step of obtaining the preset alarm rule, the electronic device may further perform the following steps:

[0130] Determine whether a user-triggered alarm rule generation request is received.

[0131] In this embodiment, the alarm rule generation request is a request for constructing an alarm rule triggered by a user through the system.

[0132] If yes, the default rule management interface is displayed.

[0133] In this embodiment, the rule management interface is a pre-built interface for assisting users in constructing alarm rules. In the rule management interface, prompt information of the rule construction process can also be displayed to guide users to perform standardized alarm rule construction processes.

[0134] Receive the rule configuration information input by the user in the rule management interface.

[0135] In this embodiment, the user can input corresponding rule configuration information in the rule management interface according to the actual alarm rule construction requirements. For example, the content of the rule configuration information may include: response time exceeds a threshold, error rate exceeds a threshold, etc.

[0136] Construct corresponding alarm rules based on the rule configuration information.

[0137] In this embodiment, the rule engine can be called to perform rule construction processing on the received rule configuration information, thereby obtaining corresponding alarm rules. In addition, users can also add, edit and delete alarm rules according to actual needs.

[0138] The alarm rules are stored in the database.

[0139] In this embodiment, the constructed alarm rules can be stored in the above database so that subsequent collectors can read and use them.

[0140] This application determines whether a request to generate an alarm rule triggered by a user is received; if so, a preset rule management interface is displayed; then the rule configuration information entered by the user in the rule management interface is received; then the corresponding alarm rule is constructed based on the rule configuration information; and the alarm rule is subsequently stored in the database. After receiving a request to generate an alarm rule triggered by a user, this application displays a rule management interface and receives the rule configuration information entered by the user in the rule management interface, and then based on the rule configuration information, alarm rules that meet user needs can be quickly and accurately constructed, thereby improving the intelligence and accuracy of alarm rule construction and improving the user experience. In addition, the alarm rules will be intelligently stored in the database, ensuring the security of the alarm rules and facilitating the subsequent collector to read and use them conveniently.

[0141] In some optional implementations of this embodiment, after step S206, the electronic device may further perform the following steps:

[0142] Perform data analysis on the link tracking data to obtain corresponding analysis data.

[0143] In this embodiment, the link tracking data can be cleaned, deduplicated, formatted and other pre-processing operations to ensure the accuracy and consistency of the data. Then, according to business needs, various analysis operations are performed on the processed link tracking data, such as calculating performance indicators such as response time, error rate, throughput, etc., performing log retrieval and correlation analysis, etc., to obtain corresponding analysis data.

[0144] Call the preset first display interface.

[0145] In this embodiment, the first display interface is a pre-designed user interface. An intuitive and easy-to-use user interface can be designed according to user needs. This user interface can display link tracking data, performance indicator data, log data, analysis data, etc., and provide various query, filtering and sorting functions.

[0146] The link tracking data and the analysis data are displayed and processed based on the first display interface.

[0147] In this embodiment, the link tracking data and the analysis data obtained after analysis and processing can be displayed in the form of charts, tables, etc. by using data visualization tools. These charts and tables should be able to clearly reflect the operating status and performance indicators of the system, so as to help users quickly understand the data and find problems.

[0148] This application obtains corresponding analysis data by performing data analysis on the link tracking data; then calls the preset first display interface; and subsequently displays and processes the link tracking data and the analysis data based on the first display interface. This application can realize visual link tracking and data analysis functions based on the use of the first display interface, obtain analysis data by performing data analysis on the link tracking data, and then display and process the link tracking data and analysis data based on the first display interface, which is conducive to greatly improving the work efficiency of developers and system administrators, and helping them better understand and optimize the operating status of the system.

[0149] In some optional implementations of this embodiment, after step S206, the electronic device may further perform the following steps:

[0150] The collected specified data is obtained from the database based on a preset timed task.

[0151] In this embodiment, the execution time and cycle of the scheduled task can be set according to actual business needs and the frequency of data collection. When the current time meets the execution time and cycle of the scheduled task, the pre-written data acquisition script is used to obtain the collected data from the database, that is, the above-mentioned specified data.

[0152] Call the preset anomaly analysis model.

[0153] In this embodiment, the construction process of the anomaly analysis model may include: selecting a suitable AI model according to business needs, such as a machine learning model (such as logistic regression, decision tree, random forest, etc.) or a deep learning model (such as convolutional neural network, recurrent neural network, etc.). Then, a data set for training the AI ​​model is filtered out from the collected data. And the data is preprocessed, such as data cleaning, feature extraction, label annotation, etc. Then, the AI ​​model is trained using the preprocessed data set, and the parameters of the model are adjusted to improve the accuracy and generalization ability of the model. Subsequently, the trained AI model is used as an anomaly analysis model and deployed to the system for real-time or batch analysis.

[0154] Anomaly analysis is performed on the designated data based on the anomaly analysis model to obtain corresponding anomaly analysis results.

[0155] In this embodiment, the specified data may be input into the abnormality analysis model, and the abnormality analysis model may be used to perform abnormality analysis on the specified data to identify possible abnormalities in the system, that is, the abnormality analysis results output by the abnormality analysis model.

[0156] Obtain a specified solution strategy corresponding to the abnormal analysis result.

[0157] In this embodiment, a solution strategy matching the abnormal analysis result can be queried from a preset abnormal solution strategy table as the above-mentioned designated solution strategy. The above-mentioned abnormal solution strategy table is a data table with targeted solution strategies corresponding to various abnormalities, which is pre-formulated based on business knowledge and experience.

[0158] The abnormality analysis result and the specified solution strategy are displayed based on a preset second display interface.

[0159] In this embodiment, the abnormal analysis results and the specified solution strategy can be displayed in the second display interface by using a data visualization tool. The construction method of the second display interface can refer to the construction method of the first display page, and no further details are given here.

[0160] The present application obtains the specified data collected from the database based on a preset timed task; then calls a preset exception analysis model; then performs exception analysis on the specified data based on the exception analysis model to obtain corresponding exception analysis results; subsequently obtains a specified solution strategy corresponding to the exception analysis result; and finally displays and processes the exception analysis result and the specified solution strategy based on a preset second display interface. The present application performs exception analysis on the specified data collected from the database by accessing the exception analysis model, and intelligently and accurately provides a specified solution strategy corresponding to the generated exception analysis result based on the use of the second display interface, which is conducive to effectively improving the reliability and stability of the system, and is conducive to helping developers and system administrators to discover and solve potential problems in a timely manner.

[0161] In some optional implementations, the user information obtained is subject to the user's consent and complies with relevant laws and policies.

[0162] In addition, the non-Company software tools or components that appear in the embodiments of the present application are merely examples and do not represent actual use.

[0163] In addition, the system of this application includes the log link tracking method function, and also supports the following extended functions:

[0164] 1. Multi-dimensional data analysis: The system not only records basic request and response data, but also collects and analyzes multi-dimensional performance indicators, such as response time, throughput, error rate, etc. Through these data, detailed performance reports can be generated to help development and operation teams better understand the operation status of the system and discover potential problems in a timely manner.

[0165] 2. Based on performance indicators and data collection, such as abnormal information such as error logs, big data analysis can be accessed to summarize and summarize the operating status of a system and where the problems are concentrated.

[0166] 3. Based on the data, AI models can also be connected to train and analyze data, so that problems and possible abnormal conditions in a system can be intelligently analyzed, and treatment plans and operating steps can be given to avoid production disasters in advance.

[0167] 4. User behavior analysis: In addition to technical tracking, the system can also record and analyze user behavior data, such as user access paths, click flows, and conversion rates. These data are crucial for optimizing user experience and improving business efficiency. They can also be provided to business systems for applicable demand development or generating BI reports.

[0168] 5. Real-time alarm: Add configurable alarm rules. When the set monitoring indicators reach the rule threshold, the abnormal situation will be sent to the operation and maintenance personnel through the configured message push method, supporting WeChat messages, SMS, emails, etc.

[0169] In addition, for developers, during the development process or troubleshooting process, they can give priority to viewing the link tracking of the log link tracking system, quickly locate a certain link, and directly go to the server to extract the log of the link according to the log identifier, so as to complete interface development and joint debugging, including problem handling, more efficiently.

[0170] For testers: In the test regression after submission, when an exception occurs, there is no need to find the developer to describe the abnormal function point and let the developer reproduce it before solving the problem as in the past. Instead, you can intuitively know which link has a problem through the web UI, and you can also extract the exception information to rule out non-code problems, and then feedback to the developer to solve the problem. If it is a non-code problem, the tester can also get information from the exception information and modify the test case, which improves the test efficiency and quality. Of course, when performing performance stress testing, it further reflects the advantages of this log link tracking system, which can count the vulnerable links of the system and avoid the occurrence of production accidents in the production environment.

[0171] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0172] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned link tracking data, the above-mentioned link tracking data can also be stored in a blockchain node.

[0173] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0174] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0175] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0176] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0177] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0178] Further references Figure 3 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a data processing device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0179] like Figure 3 As shown, the data processing device 300 described in this embodiment includes: an adding module 301, a recording module 302, a first constructing module 303, a first acquiring module 304, an assembling module 305 and a first storing module 306. Among them:

[0180] An adding module 301 is used to add a specified tag to the request header of the request based on the AOP aspect in the pre-introduced target project before the client sends a request;

[0181] A recording module 302, configured to intercept the request based on the processing component in the target project, record the specified tag corresponding to the request, and record the start time and end time of the request;

[0182] A first construction module 303, configured to construct a corresponding tracking information object based on the specified tag, the start time and the end time;

[0183] A first acquisition module 304 is used to receive the tracking information object based on a preset collector, and obtain the specified request data corresponding to the specified tag included in the tracking information object;

[0184] An assembling module 305, configured to assemble the tracking information object and the specified request data to obtain link tracking data corresponding to the request;

[0185] The first storage module 306 is used to store the link tracking data in a preset database.

[0186] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data processing method in the aforementioned implementation mode, and are not described in detail here.

[0187] In some optional implementations of this embodiment, the first acquisition module 304 includes:

[0188] A verification submodule, used to perform data verification on the tracking information object based on a preset verification strategy;

[0189] an extraction submodule, configured to extract the specified tag from the tracking information object if the tracking information object passes the data verification;

[0190] A calling submodule is used to call a preset storage structure;

[0191] A search submodule, used to search for other request data related to the specified tag from the storage structure;

[0192] The first determining submodule is configured to use the other request data as the designated request data.

[0193] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data processing method in the aforementioned implementation mode, and are not described in detail here.

[0194] In some optional implementations of this embodiment, the assembling module 305 includes:

[0195] Get submodule, used to get preset sorting strategy;

[0196] An assembling submodule, used for assembling the tracking information object and the specified request data based on the sorting strategy to obtain a complete link;

[0197] The encapsulation submodule is used to encapsulate the link to obtain a corresponding link tracking object;

[0198] The second determining submodule is used to use the link tracking object as the link tracking data.

[0199] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data processing method in the aforementioned implementation mode, and are not described in detail here.

[0200] In some optional implementations of this embodiment, the data processing device further includes:

[0201] The first calling module is used to call the preset asynchronous processing aspect;

[0202] The second acquisition module is used to obtain preset alarm rules;

[0203] A first judgment module, used to judge whether the tracking information object meets the alarm condition according to the alarm rule through the asynchronous processing aspect;

[0204] A generating module, configured to generate corresponding warning information based on the tracking information object;

[0205] The push module is used to push the alarm information to relevant personnel based on a preset message push method.

[0206] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data processing method in the aforementioned implementation mode, and are not described in detail here.

[0207] In some optional implementations of this embodiment, the data processing device further includes:

[0208] A second judgment module is used to judge whether an alarm rule generation request triggered by a user is received;

[0209] The first display module is used to display a preset rule management interface;

[0210] A receiving module, used to receive the rule configuration information input by the user in the rule management interface;

[0211] A second construction module is used to construct a corresponding alarm rule based on the rule configuration information;

[0212] The second storage module is used to store the alarm rule in the database.

[0213] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data processing method in the aforementioned implementation mode, and are not described in detail here.

[0214] In some optional implementations of this embodiment, the data processing device further includes:

[0215] A first analysis module, configured to perform data analysis on the link tracking data to obtain corresponding analysis data;

[0216] The second calling module is used to call the preset first display interface;

[0217] The second display module is used to display the link tracking data and the analysis data based on the first display interface.

[0218] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data processing method in the aforementioned implementation mode, and are not described in detail here.

[0219] In some optional implementations of this embodiment, the data processing device further includes:

[0220] A third acquisition module, used to acquire the collected specified data from the database based on a preset timed task;

[0221] The third calling module is used to call a preset abnormality analysis model;

[0222] A second analysis module, configured to perform an anomaly analysis on the designated data based on the anomaly analysis model to obtain a corresponding anomaly analysis result;

[0223] A fourth acquisition module, used to acquire a specified solution strategy corresponding to the abnormal analysis result;

[0224] The third display module is used to display the abnormality analysis result and the specified solution strategy based on a preset second display interface.

[0225] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data processing method in the aforementioned implementation mode, and are not described in detail here.

[0226] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0227] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0228] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0229] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of data processing methods, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0230] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the data processing method.

[0231] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0232] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0233] In an embodiment of the present application, before the client sends a request, a specified tag is added to the request header of the request based on the AOP aspect in the target project introduced in advance, and then the request is intercepted based on the processing component in the target project, and a corresponding tracking information object is constructed based on the recorded specified tag, start time and end time corresponding to the request. After that, the tracking information object is received based on the collector, and the specified request data corresponding to the specified tag contained in the tracking information object is obtained, and then the link tracking data corresponding to the request is assembled based on the tracking information object and the specified request data, and finally the link tracking data is stored in a preset database, so that the link tracking function of the log can be quickly and accurately implemented based on the use of the database in the future, thereby helping developers and testers to quickly locate problems, effectively reducing the processing workload of locating problems, and improving the processing efficiency of locating problems, which is conducive to improving work efficiency and work quality.

[0234] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the data processing method as described above.

[0235] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0236] In an embodiment of the present application, before the client sends a request, a specified tag is added to the request header of the request based on the AOP aspect in the target project introduced in advance, and then the request is intercepted based on the processing component in the target project, and a corresponding tracking information object is constructed based on the recorded specified tag, start time and end time corresponding to the request. After that, the tracking information object is received based on the collector, and the specified request data corresponding to the specified tag contained in the tracking information object is obtained, and then the link tracking data corresponding to the request is assembled based on the tracking information object and the specified request data, and finally the link tracking data is stored in a preset database, so that the link tracking function of the log can be quickly and accurately implemented based on the use of the database in the future, thereby helping developers and testers to quickly locate problems, effectively reducing the processing workload of locating problems, and improving the processing efficiency of locating problems, which is conducive to improving work efficiency and work quality.

[0237] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0238] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A data processing method, characterized in that: The steps include: Before the client sends a request, add a specified tag to the request header based on the AOP aspect in the target project that was previously introduced; intercepting the request based on the processing component in the target project, recording the specified tag corresponding to the request, and recording the start time and end time of the request; Constructing a corresponding tracking information object based on the specified tag, the start time and the end time; Receive the tracking information object based on a preset collector, and obtain specified request data corresponding to the specified tag included in the tracking information object; Assembling the tracking information object and the specified request data to obtain link tracking data corresponding to the request; The link tracking data is stored in a preset database.

2. The data processing method according to claim 1, characterized in that: The step of receiving the tracking information object based on the preset collector and obtaining the specified request data corresponding to the specified tag included in the tracking information object specifically includes: Performing data verification on the tracking information object based on a preset verification strategy; If the tracking information object passes the data verification, extracting the specified tag from the tracking information object; Call the preset storage structure; Searching for other request data related to the specified tag from the storage structure; The other request data is used as the designated request data.

3. The data processing method according to claim 1, characterized in that: The step of assembling the tracking information object and the specified request data to obtain the link tracking data corresponding to the request specifically includes: Get the preset sorting strategy; Assembling the tracking information object and the specified request data based on the sorting strategy to obtain a complete link; Encapsulating the link to obtain a corresponding link tracking object; The link tracking object is used as the link tracking data.

4. The data processing method according to claim 1, characterized in that: After the step of receiving the tracking information object based on the preset collector and obtaining the specified request data corresponding to the specified tag included in the tracking information object, the method further includes: Call the preset asynchronous processing aspect; Get the preset alarm rules; By means of the asynchronous processing aspect, judging whether the tracking information object meets the alarm condition according to the alarm rule; If so, generating corresponding warning information based on the tracking information object; Based on the preset message push method, the alarm information is pushed to relevant personnel.

5. The data processing method according to claim 4, characterized in that: Before the step of obtaining the preset alarm rule, the method further includes: Determine whether a user-triggered alarm rule generation request is received; If yes, display the preset rule management interface; Receiving rule configuration information input by the user in the rule management interface; Constructing corresponding alarm rules based on the rule configuration information; The alarm rules are stored in the database.

6. The data processing method according to claim 1, characterized in that: After the step of storing the link tracking data in a preset database, the method further includes: Performing data analysis on the link tracking data to obtain corresponding analysis data; Call the preset first display interface; The link tracking data and the analysis data are displayed and processed based on the first display interface.

7. The data processing method according to claim 1, characterized in that: After the step of storing the link tracking data in a preset database, the method further includes: Acquire the collected specified data from the database based on a preset timed task; Call the preset abnormal analysis model; Performing an abnormality analysis on the specified data based on the abnormality analysis model to obtain corresponding abnormality analysis results; Obtaining a specified solution strategy corresponding to the abnormal analysis result; The abnormality analysis result and the specified solution strategy are displayed based on a preset second display interface.

8. A data processing device, characterized in that: include: Add a module to add a specified tag to the request header of the request based on the AOP aspect in the pre-introduced target project before the client sends a request; A recording module, configured to intercept the request based on the processing component in the target project, record the specified tag corresponding to the request, and record the start time and end time of the request; A first construction module, configured to construct a corresponding tracking information object based on the specified tag, the start time and the end time; A first acquisition module, configured to receive the tracking information object based on a preset collector, and acquire specified request data corresponding to a specified tag included in the tracking information object; An assembling module, configured to assemble the link tracking data corresponding to the request based on the tracking information object and the specified request data; The first storage module is used to store the link tracking data in a preset database.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data processing method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data processing method according to any one of claims 1 to 7 are implemented.

Images