Message comparison method and system and computer readable storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a message comparison method and system and a computer readable storage medium, and the method comprises the steps: regularly downloading a log set from a server side, analyzing the log set by using a pre-established regular expression, and obtaining interface information and production messages of each interface; reading an interface message in the interface document, respectively extracting a node set of the interface message and a node set of the production message according to the XPath path, and comparing data structures of nodes in the interface message and the production message to obtain a message structure abnormal data set; and sending the message structure abnormal data set and the corresponding interface information to an interface verification end for accuracy verification, and modifying an interface document by using the verified message structure abnormal data set. The message comparison result is clearly displayed through the message structure abnormal data set, so that an interface principal of an interface verification end can understand problems existing in an interface document in detail and modify the interface document in time.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a message comparison method, system and computer-readable storage medium. Background Art

[0002] As the number of interfaces for civil aviation hosting services increases year by year, the number of interface documents also increases. If the interface documents are not updated in a timely manner, resulting in the inconsistency between the interface messages in the documents and the correct message structure, it will affect the progress of front-end developers in developing interfaces.

[0003] Interface messages usually refer to the data format and specifications for information exchange between two different software systems or network services. They define how to transmit data between the communicating parties in a standard, predefined way to ensure the correct understanding and application of interface information. The format and specifications of interface messages are usually determined by the back-end developer and the front-end developer through negotiation, and recorded through interface documents or other means. The interface document is used to describe the interface information provided by the system, which enables the front-end developer and the back-end developer to cooperate better, improve work efficiency, and facilitate the later personnel to view and maintain or the testers to conduct interface testing. Therefore, the interface document usually details the usage and expected behavior of each application interface API to prevent misunderstandings or errors in actual use. However, in the actual development process of the project, the message structure of the interface document is often inconsistent with the actual message structure, which directly affects the development progress of the front-end developer. Summary of the invention

[0004] The purpose of the present invention is to provide a message comparison method, system and computer-readable storage medium, which can convert messages into XPath sets and compare message data structures node by node. When the interface message in the interface document is inconsistent with the actual production message structure, the interface document can be accurately modified according to the comparison results, providing reference data for front-end developers.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A message comparison method, comprising:

[0007] Download log collection from the server at regular intervals, parse the log collection using pre-established regular expressions, and obtain interface information and production messages of each interface;

[0008] Reading the interface message in the interface document, extracting the node sets of the interface message and the production message respectively according to the XPath path, and obtaining the message structure abnormality data set by comparing the data structure of each node in the interface message and the production message;

[0009] The message structure abnormal data set and the corresponding interface information are sent to the interface verification end for accuracy verification, and then the interface document is modified using the verified message structure abnormal data set.

[0010] Furthermore, before the log collection is downloaded from the server at a scheduled time, the following is also included:

[0011] Perform information security verification on the servers where each interface is located.

[0012] Furthermore, the using of a pre-established regular expression to parse the log collection to obtain interface information and production messages of each interface includes:

[0013] Constructing a first regular expression, and using the first regular expression to match multiple production logs from the log collection;

[0014] A second regular expression is constructed, and the second regular expression is used to match the corresponding interface information and production message from each production log.

[0015] Further, the first regular expression at least includes a log start identifier, a log end identifier and a log identifier;

[0016] Using the first regular expression to match the production log from the log collection includes:

[0017] Determine the start line and the end line of the production log in the log collection by using the first regular expression, wherein the start line of the production log is determined according to the log start identifier, and the end line of the production log is determined according to the log end identifier;

[0018] Extracting a start line, then reading the log collection line by line, extracting each text line with the same log identifier as a log line, until the end line is extracted;

[0019] The extracted start line, log line, and end line are concatenated to obtain a complete production log.

[0020] Furthermore, the second regular expression includes at least the interface name, interface version, interface caller, interface software, message format and production message;

[0021] Using the second regular expression to match the corresponding interface information and production message from the production log includes:

[0022] The second regular expression is used to match the interface name, interface version, interface caller, interface software, message format and production message from the production log.

[0023] Further, the node sets of the interface message and the production message are respectively extracted according to the XPath path, and the message structure abnormality data set is obtained by comparing the data structure of each node in the interface message and the production message, including:

[0024] Selecting a node set of the interface message according to an XPath path expression, obtaining a data structure of each node in the node set of the interface message, and generating a first XPath data set;

[0025] Selecting a node set of the production message according to an XPath path expression, obtaining a data structure of each node in the node set of the production message, and generating a second XPath data set;

[0026] The first XPath data set and the second XPath data set are compared to obtain a message structure abnormality data set.

[0027] Furthermore, the comparing the first XPath data set and the second XPath data set to obtain a message structure abnormal data set includes:

[0028] traversing the first XPath data set and the second XPath data set;

[0029] If the data structure at the nodes selected along the same path is the same, skip the node;

[0030] If the data structures at the nodes selected along the same path are different, the node is marked as an abnormal node, and the abnormal node information is recorded, including the abnormal data structure, abnormal type, and the number of times the node appears;

[0031] The abnormal node information is summarized and repeated abnormal node information is deduplicated to obtain the message structure abnormal data set.

[0032] Furthermore, the method further comprises:

[0033] An interface blacklist is set, where the interface blacklist is used to record special interfaces that do not require message structure comparison.

[0034] A message comparison system, comprising:

[0035] The log download and analysis module is used to download the log collection from the server at a fixed time, parse the log collection using a pre-established regular expression, and obtain the interface information and the production messages of each interface;

[0036] A message comparison module is used to read the interface message in the interface document, extract the node sets of the interface message and the production message respectively according to the XPath path, and obtain a message structure abnormality data set by comparing the data structure of each node in the interface message and the production message;

[0037] The verification module is used to send the message structure abnormal data set and the corresponding interface information to the interface verification end for accuracy verification, and then modify the interface document using the verified message structure abnormal data set.

[0038] Based on the same inventive concept, a computer storage medium is also provided, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can implement the aforementioned message comparison method when executed.

[0039] Technical effects and advantages of the present invention:

[0040] (1) Convert the structure of the production message and the interface message into an XPath structure, and obtain an accurate message structure anomaly data set by comparing the data structure of each node in the interface message and the production message;

[0041] (2) The message structure anomaly data set clearly displays the message comparison results, so that the interface person in charge of the interface verification end can understand the problems in the interface document in detail and modify the interface document in a timely manner;

[0042] (3) It saves front-end developers time in developing interfaces and improves development efficiency.

[0043] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 Schematic diagram of the flow of the message comparison method in an embodiment of the present invention;

[0046] Figure 2 Schematic diagram of the execution flow of the message comparison method in an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the log parsing process in an embodiment of the present invention;

[0048] Figure 4 The figure is a flow chart of message structure comparison in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In order to solve the deficiencies of the prior art, the present invention discloses a message comparison method, such as Figure 1 As shown, the method includes:

[0051] Step S1, downloading a log collection from the server at a fixed time, parsing the log collection using a pre-established regular expression, and obtaining interface information and production messages of each interface;

[0052] In this step, the servers in the cluster are traversed, and the production logs of the servers where the interfaces (including the interfaces inside the program and the interfaces to the outside of the system) are located are downloaded to obtain a log collection. The interface information and the production messages of each interface are parsed from the log collection using a pre-established regular expression.

[0053] Step S2, reading the interface message in the interface document, extracting the node sets of the interface message and the production message respectively according to the XPath path, and obtaining the message structure abnormality data set by comparing the data structure of each node in the interface message and the production message;

[0054] In this step, the structures of the interface message and the production message are converted into XPath structures respectively, and the data structures of the XPath nodes are compared and the nodes with different data structures are summarized to obtain a message structure abnormality data set.

[0055] Step S3: sending the message structure abnormality data set and the corresponding interface information to the interface verification end for accuracy verification, and then modifying the interface document using the verified message structure abnormality data set.

[0056] In this embodiment, regular expressions are used to extract production messages of the interface from the log collection downloaded from the server, and the production messages and the interface messages in the interface document are converted into XPath structures respectively. By comparing the data structures between XPath nodes, both the comparison efficiency and accuracy are taken into account to obtain a message structure abnormality data set. The person in charge of the interface verification end can understand the inconsistency between the interface message and the production message structure based on the message comparison results intuitively displayed by the message structure abnormality data set, and thus modify the interface document in time, which greatly saves the development time of front-end developers and improves development efficiency.

[0057] In network communication, server information verification is a very important security measure, which is used to verify whether the server address, port, user name, encryption password and other information are empty and legal, to prevent malicious total and illegal access. Illegal or incorrect server information may be identified as illegal access or malicious attack, which will cause the connection to the server to fail and affect the download of production logs. Therefore, if Figure 2 As shown, before downloading the production log from the server, it is also necessary to perform information security verification on the server where each interface is located. On the one hand, it ensures the existence of server information and the legitimacy of the information. On the other hand, it ensures the secure connection of the server to avoid message comparison failure due to server connection timeout or connection failure.

[0058] There are multiple server clusters in the production environment of the civil aviation hosting service system. The server cluster is composed of multiple servers connected by fast communication links. When downloading production logs, traverse the servers in the cluster, obtain the list of log names under the server file directory, ensure that the downloaded files are all log files, add log name verification, download the production logs to the specified folder, and enable the folder monitoring function. After the logs in the folder are downloaded, the folder notification is triggered.

[0059] Download production logs from multiple servers where the interface is located and add them to the log collection. Since the different download speeds of each server may cause log serialization, it is necessary to extract the complete log of the interface production environment from the log collection. The embodiment of the present invention uses regular expressions to parse the log collection and uses the log messages that successfully match the regular expression rules as production logs.

[0060] Regular expressions are a tool for matching and manipulating text. They are composed of a set of common characters and metacharacters that form a regular string. Common characters include uppercase and lowercase letters, numbers, and printing symbols, while metacharacters are characters with special meanings. Regular expressions are mainly used for: pattern matching, that is, identifying strings of a specific format; string search, that is, quickly finding content that matches a specific pattern in a large amount of text; text replacement, that is, replacing strings according to the matching pattern; input verification, that is, checking whether the user input conforms to the predetermined format, etc. The matching process of regular expressions is generally: compare the regular expression with the string or the string in the text in turn. If each character matches, the match is successful. As long as there is a character that fails to match, the match is unsuccessful.

[0061] When parsing a log collection, a first regular expression is constructed, and the first regular expression is used to match multiple production logs from the log collection. The first regular expression includes at least a log start identifier, a log end identifier, and a log identifier, wherein the log start identifier is used to mark the start line of the production log, the log end identifier is used to mark the end line of the production log, and the log identifier is used to mark the log line. Any complete production log based on the civil aviation hosting service system has a specific beginning and end. For the same production log, the beginning of the log line has a unique and identical log identifier.

[0062] like Figure 3 As shown, according to a preferred embodiment, a process of matching a complete production log from a log collection using a first regular expression: using the first regular expression to determine the start line and end line of the production log in the log collection, wherein the start line of the production log is determined according to a log start identifier, and the end line of the production log is determined according to a log end identifier; after extracting the start line according to the log start identifier, reading the log collection line by line, extracting each text line with the same log identifier as a log line, until the end line is extracted according to the log end identifier; concatenating the extracted start line, log line, and end line to obtain a complete production log.

[0063] Furthermore, the corresponding interface information and production message are matched from each production log using a pre-built second regular expression, wherein the second regular expression at least includes the interface name, interface version, interface caller, message format, and production message. Therefore, the interface name, interface version, interface caller, software to which the interface belongs, message format, and production message are matched from the production log using the second regular expression.

[0064] The production message described in this embodiment can be understood as the interface message actually transmitted at the interface end. The interface message generally includes a request message and a response message. The request message is a message sent from the client to the server, and the response message is a message sent from the server to the client. The request message includes a request line, a request header, and a request body, and the response message includes a response line, a response header, and a response body. The production message includes a production request message and a production response message. The interface message in the interface document includes an interface request message and an interface response message.

[0065] In the embodiment of the present invention, the interface message in the interface document is compared with the production message from the server. If the message structures are inconsistent, the interface document needs to be corrected.

[0066] like Figure 4 As shown, before performing message comparison, it is necessary to convert the structure of the interface message in the interface document and the structure of the above-mentioned production message into an XPath structure, extract the node sets of the interface message and the production message respectively according to the XPath path, and obtain the data structure of each node, specifically:

[0067] (1) Reading interface messages in the interface document, including interface request messages and interface response messages;

[0068] (2) selecting a node set of the interface message according to an XPath path expression, obtaining a data structure of each node in the node set of the interface message, and generating a first XPath data set;

[0069] (3) Reading production messages, including production request messages and production response messages;

[0070] (4) Selecting a node set of the production message according to an XPath path expression, obtaining a data structure of each node in the node set of the production message, and generating a second XPath data set.

[0071] XPath (XML Path Language) is a language used to locate and select nodes in XML or HTML documents. XPath can select any node or node set in a document through a simple path selection syntax. Nodes refer to the basic building blocks of XML or HTML documents, which can be elements, attributes, text, etc. There are 7 types of nodes in XPath, including element node nodes, processing instruction nodes, comment nodes, and document nodes (or root nodes). Path expressions are used to locate nodes in XML or HTML documents. In addition, XPath supports complex logical relationships, such as selecting child nodes at any level, and can use wildcards and predicates for filtering. At the same time, XPath can be used in different environments, such as using XPath for data transformation in XSLT.

[0072] XPath uses path expressions to select nodes or node sets in an XML document. Nodes are selected along a path or steps. Table 1 shows the XPath path and its example analysis.

[0073]

[0074]

[0075] Table 1

[0076] Further, comparing the first XPath data set with the second XPath data set to obtain a message structure abnormal data set specifically includes:

[0077] (1) traversing the first XPath data set and the second XPath data set;

[0078] (2) If the data structures of nodes selected along the same path are the same, skip the node;

[0079] (3) If the data structures at the nodes selected along the same path are different, the node is marked as an abnormal node, and the abnormal node information is recorded, including the abnormal data structure, abnormal type, and the number of times the node appears;

[0080] (4) Summarize the abnormal node information and deduplicate the repeated abnormal node information to obtain the message structure abnormal data set.

[0081] The message structure abnormality data set records the abnormal data structure and abnormal type of the interface message and the production message at the same node. According to an embodiment of the present invention, the message structure abnormality data set and the corresponding interface information can be sent to the interface verification terminal for accuracy verification. After the verification passes, the message structure abnormality data set can be used to modify the interface document.

[0082] According to a preferred implementation, information such as interface name, interface version, interface software, interface caller, etc. are extracted based on the interface information corresponding to each abnormal node in the message structure abnormality data set, and all information is summarized and output in a table as a message comparison result set. The message comparison result set is saved on the server and can be sent to the interface person in charge by email so that the interface person in charge can check the message comparison result in time and verify its accuracy. When the structure of the interface message is inconsistent with the structure of the production message, the interface person in charge can modify the interface document in time, which greatly facilitates the work of front-end developers and improves the accuracy of interface development by front-end developers.

[0083] Moreover, for some specified interfaces, the interface messages generated by such interfaces do not need to be compared in message structure due to their particularity, and such interfaces can be added to a blacklist. Therefore, the method further includes: setting an interface blacklist, wherein the interface blacklist is used to record interfaces that do not need to be compared in message structure. During the message comparison process, the messages generated by the interfaces recorded in the interface blacklist are skipped, thereby improving the efficiency of message comparison, and the interface person in charge does not need to reconfirm such interfaces and their message structures, thereby improving the efficiency of interface inspection.

[0084] The embodiment of the present invention further provides a message comparison system, the system comprising:

[0085] The log download and analysis module is used to download the log collection from the server at a fixed time, parse the log collection using a pre-established regular expression, and obtain the interface information and the production messages of each interface;

[0086] A message comparison module is used to read the interface message in the interface document, extract the node sets of the interface message and the production message respectively according to the XPath path, and obtain a message structure abnormality data set by comparing the data structure of each node in the interface message and the production message;

[0087] The verification module is used to send the message structure abnormal data set and the corresponding interface information to the interface verification end for accuracy verification, and then modify the interface document using the verified message structure abnormal data set.

[0088] Regarding the system in the above embodiment, the specific manner in which each unit module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0089] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, including: a memory and a processor, wherein the processor is used to read and execute a computer program stored in the memory to implement the aforementioned message comparison method.

[0090] Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can implement the aforementioned message comparison method when executed.

[0091] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0092] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, the functional modules in the various embodiments of the present invention may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0093] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0094] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0095] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not used to limit the present invention. Although the present invention is described in detail with reference to the above embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the above embodiments, or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A message comparison method, characterized in that: The method comprises: Download log collection from the server at regular intervals, parse the log collection using pre-established regular expressions, and obtain interface information and production messages of each interface; Reading the interface message in the interface document, extracting the node sets of the interface message and the production message respectively according to the XPath path, and obtaining the message structure abnormality data set by comparing the data structure of each node in the interface message and the production message; The message structure abnormal data set and the corresponding interface information are sent to the interface verification end for accuracy verification, and then the interface document is modified using the verified message structure abnormal data set.

2. The method according to claim 1, characterized in that: Before downloading the log collection from the server at a scheduled time, it also includes: Perform information security verification on the servers where each interface is located.

3. The method according to claim 1, characterized in that The using of a pre-established regular expression to parse the log collection and obtain interface information and production messages of each interface includes: Constructing a first regular expression, and using the first regular expression to match multiple production logs from the log collection; A second regular expression is constructed, and the second regular expression is used to match the corresponding interface information and production message from each production log.

4. The method according to claim 3, characterized in that: The first regular expression at least includes a log start identifier, a log end identifier and a log identifier; Using the first regular expression to match the production log from the log collection includes: Determine the start line and the end line of the production log in the log collection by using the first regular expression, wherein the start line of the production log is determined according to the log start identifier, and the end line of the production log is determined according to the log end identifier; Extracting a start line, then reading the log collection line by line, extracting each text line with the same log identifier as a log line, until the end line is extracted; The extracted start line, log line, and end line are concatenated to obtain a complete production log.

5. The method according to claim 3 or 4, characterized in that: The second regular expression includes at least the interface name, interface version, interface caller, interface software, message format and production message; Using the second regular expression to match the corresponding interface information and production message from the production log includes: The second regular expression is used to match the interface name, interface version, interface caller, interface software, message format and production message from the production log.

6. The method according to claim 1, characterized in that The node sets of the interface message and the production message are respectively extracted according to the XPath path, and the message structure abnormality data set is obtained by comparing the data structure of each node in the interface message and the production message, including: Selecting a node set of the interface message according to an XPath path expression, obtaining a data structure of each node in the node set of the interface message, and generating a first XPath data set; Selecting a node set of the production message according to an XPath path expression, obtaining a data structure of each node in the node set of the production message, and generating a second XPath data set; The first XPath data set and the second XPath data set are compared to obtain a message structure abnormality data set.

7. The method according to claim 6, characterized in that The comparing the first XPath data set and the second XPath data set to obtain a message structure abnormality data set includes: traversing the first XPath data set and the second XPath data set; If the data structure at the nodes selected along the same path is the same, skip the node; If the data structures at the nodes selected along the same path are different, the node is marked as an abnormal node, and the abnormal node information is recorded, including the abnormal data structure, abnormal type, and the number of times the node appears; The abnormal node information is summarized and repeated abnormal node information is deduplicated to obtain the message structure abnormal data set.

8. The method according to claim 1, characterized in that: The method further comprises: An interface blacklist is set, where the interface blacklist is used to record special interfaces that do not require message structure comparison.

9. A message comparison system, characterized in that: The system comprises: The log download and analysis module is used to download the log collection from the server at a fixed time, parse the log collection using a pre-established regular expression, and obtain the interface information and the production messages of each interface; A message comparison module is used to read the interface message in the interface document, extract the node sets of the interface message and the production message respectively according to the XPath path, and obtain the message structure abnormality data set by comparing the data structure of each node in the interface message and the production message; The verification module is used to send the message structure abnormal data set and the corresponding interface information to the interface verification end for accuracy verification, and then modify the interface document using the verified message structure abnormal data set.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed, the message comparison method according to any one of claims 1 to 8 is implemented.

Images