Data processing method, device, equipment and storage medium
By obtaining and building server messages, simulating sending to the client, reproducing exception scenarios, solving the problem of inefficient abnormal positioning communication between the client and the server, and achieving efficient and accurate abnormal positioning.
Patent Information
- Application Number
- CN202011116457.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-19
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-10-19
AI Technical Summary
In Internet applications, the abnormal positioning of data communication between the client and the server is inefficient. The existing technology requires manual analysis of log information, which is complex and poor in efficiency, making it difficult to accurately locate the cause of the abnormality.
By obtaining the log information received by the client from the server, building server messages and simulating sending to the client, reproducing the exception scenario, thereby quickly performing exception positioning.
It improves the efficiency and accuracy of abnormal positioning, reduces the difficulty of abnormal positioning, can quickly reproduce abnormal scenes, and improves positioning accuracy.
Smart Images

Figure CN112199229B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of Internet technology, and in particular, to a data processing method, apparatus, device and storage medium. Background Art
[0002] In Internet applications, clients and servers usually communicate frequently and in large quantities. During the communication process between the client and the server, the client often encounters various exceptions or errors, and some errors are caused by the data sent by the server.
[0003] After an exception occurs on the client, the cause of the exception is usually found to locate the exception. In the prior art, technicians usually first determine the abnormal time when the client is abnormal, and check the log information before and after the abnormal time. The log information records the messages communicated between the client and the server. The technicians analyze the log information to infer the cause of the abnormality on the client. However, technicians usually need to analyze the log information based on personal experience to determine the cause of the abnormality, resulting in poor efficiency in locating the abnormality. Summary of the invention
[0004] The embodiments of the present application provide a data processing method, apparatus, device and storage medium to improve the efficiency and accuracy of abnormality location.
[0005] In a first aspect, the present application provides a data processing method, comprising:
[0006] Obtaining first log information, where the first log information includes content of a message received by the client from the server;
[0007] Determine a server message according to the message content included in the first log information, and send the server message to the client;
[0008] Execute corresponding client operations according to the server message.
[0009] In a second aspect, the present application provides a data processing device, including:
[0010] An acquisition module, configured to acquire first log information, wherein the first log information includes message content received by the client from the server and attribute information corresponding to the message content;
[0011] A determination module, configured to determine a server message according to the message content included in the first log information;
[0012] A sending module, used for sending the server message to the client;
[0013] The processing module is used to execute corresponding client operations according to the server message.
[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any one of the methods described in the first aspect.
[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, including:
[0016] Processor; and
[0017] A memory, configured to store executable instructions of the processor;
[0018] The processor is configured to perform the method according to any one of the first aspects by executing the executable instructions.
[0019] The data processing method, apparatus, device and storage medium provided in the embodiments of the present application obtain log information, which includes the message content received by the client from the server, construct a server message according to the message content included in the log information, and simulate sending the server message to the client; perform corresponding client operations according to the server message to reproduce the abnormal scenario, and then quickly locate the abnormality according to the abnormal scenario, which is highly efficient, reduces the difficulty of abnormality location, and has high accuracy in abnormality location. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0021] Figure 1 A schematic diagram of the system architecture provided for an embodiment of the present application;
[0022] Figure 2 It is a flowchart of an embodiment of a data processing method provided by the present application;
[0023] Figure 3 This is a schematic diagram of a simulation tool interface of a method embodiment provided by the present application;
[0024] Figure 4 It is a structural schematic diagram of an embodiment of a data processing device provided by the present application;
[0025] Figure 5 It is a schematic diagram of the structure of an electronic device embodiment provided by the present application.
[0026] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0028] The terms "including" and "having" and any variations thereof in the specification and claims of the present application and the drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.
[0029] First, the application scenarios involved in the embodiments of the present application are introduced.
[0030] Figure 1 This is a schematic diagram of the system architecture provided in the embodiment of the present application. Figure 1 As shown, the system architecture of the embodiment of the present application may include but is not limited to: an electronic device 11 and a server 12. The electronic device 11 includes, for example, user devices such as a mobile phone, a tablet computer, and a personal computer.
[0031] The electronic device 11 and the server 12 may be connected via a network.
[0032] Figure 1 An electronic device is used as an example for explanation. The electronic device can be used as a client-side device of an Internet application.
[0033] The method provided in the embodiment of the present application can be implemented by an electronic device such as a processor of the electronic device executing a corresponding software code, or can be implemented by the electronic device executing the corresponding software code while performing data interaction with a server.
[0034] Among the current Internet applications, most are networked applications. Clients and servers are communicating frequently and in large quantities. During the development and testing of client software, or when using it, various exceptions or errors often occur. Some errors are caused by server data, such as client exceptions and errors caused by messages sent by the server, such as interface data errors, interface disorder, special effects playback errors, etc.
[0035] In the related art, the cause of the exception is located by manually analyzing the log file. The client's input and output operations will mostly communicate with the server. The log file will record the protocol messages of the communication between the client and the server, and record the places where the exception may occur, such as the time when the exception occurred, etc. Then the programmer will gradually analyze the data log and exception log recorded in the log file to locate the cause of the exception.
[0036] Since the client generates a large amount of log information when communicating with the server, this greatly increases the difficulty of log analysis, and is very time-consuming and inefficient; secondly, when locating anomalies, it is usually necessary to manually analyze the log files, and manually simulate the abnormal triggering process by testers or designers to reproduce the abnormal scene and then locate the anomaly. This method is complicated to operate and easily deviates from the actual abnormal occurrence process, and it is impossible to reproduce the scene, resulting in poor accuracy in locating the anomaly. Therefore, in order to quickly locate the cause of the anomaly, the embodiment of the present application provides a data processing method that can quickly and conveniently reproduce the abnormal scene and locate the anomaly.
[0037] The technical concept of the method of the embodiment of the present application is as follows:
[0038] By obtaining log information, which includes the message content received by the client from the server, a server message is constructed based on the log information, and the sending of the server message to the client is simulated; the corresponding client operation is performed according to the server message to reproduce the abnormal scenario, and then the abnormality is located according to the abnormal scenario, with high efficiency and accuracy.
[0039] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0040] Figure 2 1 is a flow chart of an embodiment of a data processing method provided by the present application. Figure 2 As shown, the method provided in this embodiment, the execution subject of the method may be the electronic device where the client is located or other electronic devices, and the method includes:
[0041] Step 101: Obtain first log information, where the first log information includes message content received by the client from the server.
[0042] Specifically, when the client is running, it usually records the messages sent by the server and generates log information, such as saving it in a log file;
[0043] The log information includes logs of multiple messages sent by the server.
[0044] Optionally, the log format of a message can be defined as follows:
[0045] [Main protocol number, sub-protocol number][Business module name][Timestamp][Message result][Message entity].
[0046] Among them, the main protocol number and sub-protocol number are the protocol identification IDs defined for communication between the client and the server, the business module name corresponds to the specific business module of the client, the timestamp is the timestamp when the client receives the server message, and the message result is the message identification code returned by the server. For example, 0 indicates that the message is correct, 1 indicates an invalid request, 2 indicates a parameter error, etc. The message entity is the specific message content returned, which can be a lightweight data exchange format (JavaScriptObject Notation, JSON) object or other defined format content.
[0047] Among them, the attribute information corresponding to the message content includes at least one of the following: the main protocol number, the sub-protocol number, the business module corresponding to the message content, the timestamp of the message content, and the message identification code, that is, the items included in the above attribute information are optional, that is, the log format of a message includes at least one of the message content and attribute information.
[0048] The first log information is obtained from the recorded log information, and the first log information includes the message content received by the client from the server and the attribute information corresponding to the message content, such as selecting the log information of a certain time period, or selecting the log information corresponding to a certain business module, etc. For example, the log information of the time period in which the abnormality occurs, or the log information of the business module in which the abnormality occurs is selected. The method of the embodiment of the present application can be implemented by a log parsing simulation tool, and the log parsing simulation tool can be an application.
[0049] In one embodiment, the client program may be set to debug mode before step 101. After the client program is set to debug mode, the operation of the log parsing simulation tool may be triggered. The log parsing simulation tool may have a display interface, and the user simulates the reproduction of the abnormal scene through the display interface. For example, log information is loaded through the log parsing simulation tool, for example Figure 3As shown, click the Select Log button to load log information from a certain log path to simulate the reproduction of abnormal scenarios.
[0050] After the log information is loaded, the tool parses the log information and parses each line of string in the log information to obtain the main information of each message content, including the main protocol number, sub-protocol number, message identification code, message content, received timestamp, and corresponding business module, such as business module name. Optionally, a mapping relationship can be established between the message content and the corresponding business module.
[0051] After parsing is completed, some information obtained from the parsing will be displayed on the tool interface, such as the name of the business module corresponding to the log information under the path, the timestamps of the first and last log information received by the corresponding business module, etc.
[0052] The first log information is obtained from the log information, for example, log information of a certain business module or a certain time period is selected.
[0053] Step 102: Determine a server message according to the message content included in the first log information, and send the server message to the client.
[0054] Specifically, a server message is constructed according to the acquired first log information, for example, a server message conforming to a protocol format is constructed. The server message may be one or more.
[0055] Specifically, the first log information can be parsed to obtain at least one message content in the first log information, and then a server message corresponding to each message content can be constructed based on the at least one message content. For example, the server message can be constructed according to the protocol format for communication between the server and the client. The server message includes, for example, at least one of the following: main protocol number, sub-protocol number, business module name, message result and message entity, etc.
[0056] Furthermore, the attribute information corresponding to each message content can also be obtained, such as timestamp, business module corresponding to the message content, main protocol number, sub-protocol number, message identification code, etc. When constructing the server message, at least one message content and the attribute information corresponding to each message content can be used.
[0057] Simulate sending the server message to the client.
[0058] Step 103: Execute corresponding client operations according to the server message.
[0059] Specifically, when the client receives the simulated server message, it triggers the execution of the corresponding client operation. For example, the client operation includes at least one of the following: page refresh, special effect playback, and page error reporting.
[0060] Executing client operations will trigger the client to respond, realizing the reproduction of abnormal scenarios. For example, abnormal client errors caused by server messages, interface data errors, interface disorder, special effects playback errors, etc. can be displayed in time, thus completing the reproduction of abnormal scenarios.
[0061] The method of this embodiment obtains log information, which includes the message content received by the client from the server, constructs a server message according to the message content included in the log information, and simulates sending the server message to the client; performs corresponding client operations according to the server message to reproduce the abnormal scenario, and then can quickly locate the abnormality according to the abnormal scenario, which is highly efficient, reduces the difficulty of abnormality location, and has high accuracy in abnormality location.
[0062] On the basis of the above embodiment, further, the method of this embodiment further includes:
[0063] According to the result of the client operation, abnormality positioning is performed.
[0064] Specifically, each server message will trigger the response of the corresponding business module of the client, that is, execute the corresponding client operation, and these client operations will generate corresponding feedback, such as client abnormal error caused by server message data, interface data error, interface disorder, special effect playback error, etc., which can be displayed in time, that is, the scene is reproduced. In this way, the server message that caused the abnormality is located.
[0065] For example, the exception location result information can also be generated, such as the corresponding server message when the exception occurs, the corresponding business module, the executed client operation and other information.
[0066] In one embodiment, step 101 may be implemented in the following manner:
[0067] Determine simulation parameters, wherein the simulation parameters include at least one of the following: an identifier of a target service module, or a simulation period;
[0068] The first log information is obtained from the second log information according to the simulation parameters and the attribute information corresponding to the content of each message in the second log information.
[0069] Specifically, in order to improve processing efficiency, it is not necessary to reproduce the scene of the message content in all the log information recorded by the client, that is, only part of the log information needs to be obtained to reconstruct the scene.
[0070] First, simulation parameters need to be determined, such as business modules where exceptions often occur, business modules where exceptions have occurred recently, or time periods when exceptions are likely to occur. The time periods when exceptions are likely to occur in different business modules may be different.
[0071] That is, the simulation parameters include at least one of the following: an identifier of a target service module and a simulation period.
[0072] For example, the time period from 6-311:28 to 6-317:00 is selected as the simulation period.
[0073] The selected simulation period can be adjusted according to the time period corresponding to the scene to be reproduced, to avoid selecting too long a time period which results in too long a scene reproduction time.
[0074] In one embodiment, the simulation parameters include an identifier of a target service module and a simulation period, and the message content included in the first log information is: the message content corresponding to the target service module and received by the client during the simulation period.
[0075] For example, you can select the target business module to be simulated in the log parsing simulation tool, set the time period to be simulated, and press the simulation button of the tool. The tool will automatically obtain the log information corresponding to the simulation parameters, and then build a server-side message and simulate sending it to reproduce the abnormal scenario.
[0076] like Figure 3 As shown in the figure, after log parsing is completed, the log information after parsing will be refreshed to the tool interface. It mainly includes the business module name, the timestamp of the first log information and the last log information received by the corresponding business module.
[0077] In the above implementation, the log of communication messages between the client and the server can be automatically parsed to obtain log information for scene reproduction. The log information includes the message content received by the client from the server, so as to reproduce the abnormal scene according to the log information, and then quickly locate the abnormality according to the abnormal scene, which is more efficient, reduces the difficulty of abnormality locating, and has higher accuracy in abnormality locating.
[0078] In one embodiment, "determining the server-side message" in step 102 may be implemented in the following manner:
[0079] Parsing the first log information to obtain at least one message content and attribute information corresponding to each message content;
[0080] According to at least one message content and attribute information corresponding to each message content, a server-side message corresponding to each message content is determined.
[0081] Specifically, the log of each message in the first log information is parsed to obtain the message content of each message and the attribute information corresponding to each message content, such as the timestamp, the business module corresponding to the message content, the main protocol number, the sub-protocol number, the message identification code, etc.
[0082] According to each message content and the attribute information corresponding to each message content, a server message corresponding to each message content is generated in a corresponding format, for example, according to the format of a corresponding protocol for server-client communication.
[0083] In one embodiment, the number of server messages is multiple, and "sending the server message to the client" in step 102 can be implemented as follows:
[0084] Send multiple server messages to the client at preset time intervals.
[0085] Specifically, the preset time interval can be determined according to the timestamp of each message content in the first log information, that is, the time interval between adjacent server messages is calculated according to the timestamps of adjacent server messages, or it can be based on the default time interval. For example, the message distribution time interval can be set to send 10 messages every 30 milliseconds.
[0086] The server messages are distributed in a time-sharing simulation according to the time intervals of adjacent server messages.
[0087] The corresponding business module responds after receiving the server message, that is, executes the corresponding client operation, and then makes corresponding feedback based on the executed client operation, such as refreshing the interface, playing special effects, etc. If an exception occurs, it can be directly displayed, that is, the scene reproduction is completed.
[0088] In the above implementation, a server message is constructed based on the log information obtained through analysis, and the server message is simulated to be sent to the client; the corresponding client operation is performed according to the server message to restore the client scenario and reproduce the abnormal scenario, so that the abnormality can be quickly located according to the abnormal scenario, which is highly efficient, reduces the difficulty of abnormality location, and has a high accuracy in abnormality location.
[0089] Figure 4 A structural diagram of an embodiment of a data processing device provided by the present application, such as Figure 4 As shown, the data processing device of this embodiment includes:
[0090] The acquisition module 401 is used to acquire first log information, where the first log information includes the content of a message received by the client from the server;
[0091] A determination module 402, configured to determine a server message according to the message content included in the first log information;
[0092] A sending module 403 is used to send the server message to the client;
[0093] The processing module 404 is used to execute corresponding client operations according to the server message.
[0094] In a possible implementation, the acquisition module 401 is specifically configured to:
[0095] Determining simulation parameters, wherein the simulation parameters include at least one of the following: an identifier of a target service module and a simulation period;
[0096] The first log information is obtained from the second log information according to the simulation parameters and the attribute information corresponding to the content of each message in the second log information.
[0097] In a possible implementation, the simulation parameters include an identifier of a target service module and a simulation period, and the message content included in the first log information is: the message content corresponding to the target service module and received by the client during the simulation period.
[0098] In a possible implementation, the attribute information includes at least one of the following information: a timestamp of the message content, a service module corresponding to the message content, a main protocol number, a sub-protocol number, and a message identification code.
[0099] In a possible implementation, the determination module 402 is specifically configured to:
[0100] Parsing the first log information to obtain at least one message content and attribute information corresponding to each of the message contents;
[0101] According to the at least one message content and the attribute information corresponding to each of the message contents, a server-side message corresponding to each of the message contents is determined.
[0102] In a possible implementation, the number of the server-side messages is multiple; the sending module 403 is specifically used to:
[0103] Send multiple server messages to the client at preset time intervals.
[0104] In a possible implementation, the processing module 404 is further configured to:
[0105] According to the result of the client operation, abnormality positioning is performed.
[0106] The device of this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar and will not be repeated here.
[0107] Figure 5 The structure diagram of the electronic device embodiment provided in this application is as follows: Figure 5 As shown, the electronic device includes:
[0108] A processor 501 , and a memory 502 for storing executable instructions of the processor 501 .
[0109] Optionally, it may also include: the input device 503 can receive input data, such as receiving a server message, and the output device 504 can send data to other devices, such as sending data to a server.
[0110] The above components may communicate via one or more buses.
[0111] Among them, the processor 501 is configured to execute the corresponding method in the aforementioned method embodiment by executing the executable instructions. The specific implementation process can be found in the aforementioned method embodiment and will not be repeated here.
[0112] A computer-readable storage medium is also provided in an embodiment of the present application, on which a computer program is stored. When the computer program is executed by a processor, the corresponding method in the aforementioned method embodiment is implemented. The specific implementation process can be found in the aforementioned method embodiment. The implementation principle and technical effects are similar and will not be repeated here.
[0113] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0114] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A data processing method, characterized in that: The method comprises: Obtaining first log information, where the first log information includes content of a message received by the client from the server; Constructing a server message corresponding to each message content according to the message content and the attribute information corresponding to the message content included in the first log information, and sending the server message to the client; Execute a corresponding client operation according to the server message, determine the result of the client operation, the result includes a client exception error, if it is determined that the client exception error is caused by the server message data, then locate the exception of the server data message.
2. The method according to claim 1, characterized in that Get the first log information, including: Determining simulation parameters, wherein the simulation parameters include at least one of the following: an identifier of a target service module and a simulation period; The first log information is obtained from the second log information according to the simulation parameters and the attribute information corresponding to the content of each message in the second log information.
3. The method according to claim 2, characterized in that The simulation parameters include an identifier of a target service module and a simulation period. The message content included in the first log information is: the message content corresponding to the target service module and received by the client during the simulation period.
4. The method according to claim 2 or 3, characterized in that: The attribute information includes at least one of the following information: a timestamp of the message content, a service module corresponding to the message content, a main protocol number, a sub-protocol number, and a message identification code.
5. The method according to any one of claims 1 to 3, characterized in that: Constructing a server message corresponding to each message content according to the message content and the attribute information corresponding to the message content included in the first log information, including: Parsing the first log information to obtain at least one message content and attribute information corresponding to each of the message contents; According to the at least one message content and the attribute information corresponding to each of the message contents, a server-side message corresponding to each of the message contents is determined.
6. The method according to claim 5, characterized in that The number of the server-side messages is multiple; sending the server-side messages to the client includes: Send multiple server messages to the client at preset time intervals.
7. A data processing device, characterized in that: include: An acquisition module, configured to acquire first log information, wherein the first log information includes content of a message received by the client from the server; A determination module, configured to construct a server-side message corresponding to each message content according to the message content and the attribute information corresponding to the message content included in the first log information; A sending module, used for sending the server message to the client; A processing module is used to perform corresponding client operations according to the server message, determine the result of the client operation, and the result includes a client exception error. If it is determined that the client exception error is caused by the server message data, the server data message is abnormally located.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 6 by executing the executable instructions.
10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.
