A method for analyzing and adjusting interface data structure based on Huffman tree group

Abstract

The present invention belongs to the field of data processing technology, and in particular relates to a method for analyzing and adjusting interface data structures based on a Huffman tree group. The method comprises: collecting data fields from a collected system and a collecting system, using one data field as a child node of a Huffman tree and the other data field as a root node of the Huffman tree to determine a Huffman tree. After determining a plurality of Huffman trees, the data fields are grouped into a Huffman tree group and stored on a disk. The Huffman tree is then evolved. During the evolution process, if the collected system fails to retrieve a matching field when attempting to hit a node in the Huffman tree, a log record is generated, periodically extracted, and an alarm is issued. The present invention can solve the problem in the prior art of collecting and frequency analyzing unstructured data, which has become a barrier to the development of business systems.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to an interface data structure analysis and adjustment method based on a Huffman tree group. Background Art

[0002] At present, the business system is based on system monitoring in the form of interfaces. In each business system, the data structure for data embedding consists of unstructured data. However, since the request message structure required in the monitoring system is a structured field, when the monitoring system receives the embedded data from each business system, it is necessary to consider converting the messages of each business system to resolve the data differences.

[0003] In existing technologies, the development of tracking data for various business systems is subject to numerous uncertainties, including the erratic frequency of tracking data and complex message structures. Consequently, the entire tracking data development process requires significant investment in manpower and effort, and involves lengthy maintenance cycles, significantly impacting the development of these business systems. Summary of the Invention

[0004] The technical problem solved by the present invention is to provide an interface data structure analysis and adjustment method based on a Huffman tree, so as to solve the problem in the prior art that the collection and frequency analysis of unstructured data affects the development of business systems.

[0005] The basic solution provided by the present invention is a method for analyzing and adjusting interface data structure based on a Huffman tree group, comprising:

[0006] Construct a Huffman tree, define the data field of the acquisition system as the root node, define the data field of the acquired system as the child node, initialize the data field, construct several Huffman trees, and group the several Huffman trees into a set to form a Huffman tree group;

[0007] Huffman tree group structure storage: cache the constructed Huffman tree group in the source system memory and periodically persist the Huffman tree group structure to disk;

[0008] The evolution of the Huffman tree structure. If the data field of the matching node cannot be retrieved when the collection system requests to hit the Huffman tree node field, the collection system information will be logged;

[0009] Unmatched data field alarm processing: unmatched log records recorded during the evolution of the Huffman tree structure are periodically retrieved, and an alarm is issued when new log records appear.

[0010] The principle and advantage of the present invention are: first, the data fields of the collected system and the collecting system are collected, one as the child node of the Huffman tree, and the other as the root node of the Huffman tree, thereby determining a Huffman tree. After determining several Huffman trees, they are grouped into a Huffman tree group and stored in a disk. Then, the evolution of the Huffman tree is implemented. During the evolution process, if the collected system fails to retrieve the matching field when trying to hit the node of the Huffman tree, a log record is generated, and periodic extraction and alarm reminder are issued. Therefore, the advantages of the present invention are: (1) when constructing the Huffman tree, the required data fields are collected by automatic collection, and the collection method is in a relatively stable state. Therefore, there is no need to adjust the Huffman tree during the collection process; (2) in the evolution process of the Huffman tree, the automatic retrieval method is used to minimize the manual intervention cost of the collection system to the collected system interface, greatly reducing the research and development cost; (3) through the above two advantages, the problem that the existing technology of collecting and frequency analyzing unstructured data will affect the development of the collected system is solved.

[0011] Furthermore, the construction of the Huffman tree also includes:

[0012] Weight data collection, including obtaining weight data of the collection system and the collected system;

[0013] Initializing the Huffman tree includes initializing the data fields and weight data of the acquisition system and the acquired system, and determining a Huffman tree.

[0014] Beneficial effect: After the weight data of the collecting system and the collected system are collected through the weight data collection step, the Huffman tree is initialized, and finally a Huffman tree transmitted by the data interface is obtained, which is conducive to the formation of the Huffman tree.

[0015] Furthermore, the weight data collection also includes:

[0016] User preset acquisition, receiving user input or receiving user-set weight data, and configuring it in the database;

[0017] System acquisition, including the automatic creation and adjustment of data weights by the acquisition system during the acquisition process.

[0018] Beneficial effects: Weight data is obtained from both user presets and the system, with both manual and automatic data acquisition, meeting the richness of data sources and data transmission.

[0019] Furthermore, the initialization of the Huffman tree also includes:

[0020] Null value processing, including if there is no weight data in the collected system, the data fields and weight data of the collecting system and the collected system are initialized into an empty Huffman tree;

[0021] The weight data processing includes selecting a node of the weight data as an initialization node and determining a Huffman tree if weight data exists in the acquisition system and the acquired system.

[0022] Beneficial effect: By initializing the fields without weighted data in the collected data, the fields with weighted data in the collected data become a node of the Huffman tree, avoiding the initialization of preset unweighted data during the evolution of the Huffman tree, which is beneficial to the evolution of the Huffman tree structure.

[0023] Furthermore, the evolution of the Huffman tree structure also includes:

[0024] Huffman tree structure optimization includes initializing weight intervention on data fields based on log records, rebuilding the Huffman tree, adding 1 to the weight data of the node, and checking whether the Huffman tree structure needs to be adjusted at this time.

[0025] Beneficial effects: Based on the characteristics of the Huffman tree, the situation where field hit request mismatch occurs in the Huffman tree is optimized. In the worst case, the field hit time complexity is 0 (logN); under normal circumstances, the system's weight for the same field is basically stable, and the hit time complexity is 1. In this way, by changing the weight data of the node, its hit time complexity is increased by 1, so that the structure of the Huffman tree remains stable.

[0026] Furthermore, the collected system information in the evolution of the Huffman tree structure includes a data field, a name of the collected system, and request interface information.

[0027] Beneficial effect: The data fields, system names and request interface information of the collected systems are displayed to the user, making it easier for the user to fully understand the unmatched information.

[0028] Furthermore, the alarm prompt method in the unmatched data field alarm processing is one or more of a DingTalk message prompt or an email prompt.

[0029] Beneficial effects: Users can be reminded in a timely manner through DingTalk or email. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flowchart of an embodiment of the present invention;

[0031] Figure 2 Schematic diagram of a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0032] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. At the same time, although a logical order is shown in the embodiments, in some cases, the steps shown or described may be performed in an order different from that shown here.

[0033] The following is further described in detail through specific implementation methods:

[0034] The embodiment is basically as shown in the attached Figure 1 and Figure 2 As shown: an interface data structure analysis and adjustment method based on a Huffman tree group is applied to the data field and data weight analysis between the acquisition system based on the interface form and the collected system. Specifically, when performing data field conversion between the business system and its monitoring system connected in the form of an interface, the business system serves as the collected system, and the monitoring system serves as the acquisition system. The data fields required by the monitoring system are structured fields, while the data structure of the business system for burying points is composed of unstructured fields. In order to solve the data differences between the monitoring system and the business system, when the monitoring system receives the buried point data of the business system, the unstructured fields of the business system must be converted into structured fields. Therefore, the present invention adopts a message structure optimization method based on a Huffman tree group to solve the above problems.

[0035] Step 1: Construct a Huffman tree. Define the data field of the acquisition system as the root node, define the data field of the acquired system as the child node, initialize the data field, construct several Huffman trees, and group the several Huffman trees into a set to form a Huffman tree group. Specifically, first obtain the weight data of the acquisition system and the acquired system, that is, the weight data of the monitoring system and the business system. There are two ways to obtain it. The first is to receive user input or user-set weight data and configure it into the database. In this embodiment, a batch of data samples is first added by the system receiving active user operations, and the system data fields with higher weights are selected as the first batch of Huffman tree weight data. Then, the weight data in the acquired system is collected. During the collection process, only the weight data of the acquisition system needs to be added for the unmapped fields. The second method is for the acquisition system to automatically create and adjust the data weights during the collection process.

[0036] Then initialize the Huffman tree group, including initializing the data fields and weight data of the acquisition system and the collected system, and determine a Huffman tree. The initialization process includes two types. The first is when there is no weight data in the collected system, then the data fields and weight data of the acquisition system and the collected system are initialized to an empty Huffman tree; the second is if there is weight data in the acquisition system and the collected system, then select a node of the weight data as the initialization node to determine a Huffman tree.

[0037] Step 2: Huffman tree group structure storage, cache the constructed Huffman tree group in the source system memory, and periodically persist the Huffman tree group structure to the disk. In this embodiment, in order to avoid the problems of redundant and unstable storage, the storage of the Huffman tree group structure is carried out by traversing the Huffman tree nodes layer by layer. Only 2N-1 bits are needed to store the entire tree structure, where N represents the total number of Huffman tree nodes, and when the total number of nodes is greater than 2, there is no need to record the tree structure. This method can save space for storing the Huffman tree, prevent waste of storage space, and will not cause the problem of unstable storage length.

[0038] Step 3: Evolution of the Huffman tree structure. After the Huffman tree is constructed, when the data field conversion is performed between the collection system and the collected system, the weight data of the collected system continuously sends hit requests to the collection system. If the collected system cannot retrieve a matching node after requesting to hit the Huffman tree node field, it indicates that the field node weight exceeds the root node or child node in the Huffman tree, and the tree structure needs to be optimized. The system then logs the information of the collected system that failed to hit the request. The information of the collected system includes data fields, the name of the collected system, and the request interface information.

[0039] Due to the characteristics of the Huffman tree structure, in the worst case, the hit time complexity of a field is 0 (LogN); under normal circumstances, the target system's weights for the same field are basically stable, and the hit time complexity is 0. When optimizing the tree structure, first retrieve the log record to determine whether the data field needs to be initialized with weight intervention. If so, add 1 to the weight data of the node, and check again whether the Huffman tree structure still needs to be adjusted. This process is repeated in a loop. In this way, the field content of the Huffman tree will gradually become richer, and the weights and system scenarios will be relatively stable. In business systems, when system changes are relatively infrequent, the weight ratio of system access instructions received by the system is relatively stable. For example, during a transaction, the weight of the payment scenario is usually higher than the weight of the refund scenario. Therefore, the weight of the collected data will show a relatively stable weight record under the corresponding system.

[0040] Step 4: Unmatched data structure alarm processing, including periodic retrieval of unmatched log records recorded during the evolution of the Huffman tree structure, and issuing an alarm prompt when new log records appear. This step is when the collected system request fails to hit the Huffman tree structure and is recorded in the log record. The user needs to view it, so an alarm is set. The alarm method is DingTalk reminder or email reminder.

[0041] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0042] The above are only embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme are not described in detail here. Ordinary technicians in the field are aware of all common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the field can improve and implement this scheme in combination with their own abilities under the inspiration given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A method for analyzing and adjusting interface data structure based on Huffman tree group, characterized in that: include: Construct a Huffman tree, define the data field of the acquisition system as the root node, define the data field of the acquired system as the child node, initialize the data field, construct several Huffman trees, and group the several Huffman trees into a set to form a Huffman tree group; Huffman tree group structure storage: cache the constructed Huffman tree group in the source system memory and periodically persist the Huffman tree group structure to disk; The evolution of the Huffman tree structure. If the data field of the matching node cannot be retrieved when the collection system requests to hit the Huffman tree node field, the collection system information will be logged; Unmatched data field alarm processing: periodically retrieve unmatched log records recorded during the evolution of the Huffman tree structure, and issue an alarm when new log records appear; The evolution of the Huffman tree structure also includes: Huffman tree structure optimization includes initializing weight intervention on data fields based on log records, rebuilding the Huffman tree, adding 1 to the weight data of the node, and checking whether the Huffman tree structure needs to be adjusted at this time.

2. The method for analyzing and adjusting interface data structure based on a Huffman tree group according to claim 1, wherein: The construction of the Huffman tree also includes: Weight data collection, including obtaining weight data of the collection system and the collected system; Initializing the Huffman tree includes initializing the data fields and weight data of the acquisition system and the acquired system, and determining a Huffman tree.

3. The method for analyzing and adjusting interface data structure based on a Huffman tree group according to claim 2, wherein: The weight data collection also includes: User preset acquisition, receiving user input or receiving user-set weight data, and configuring it in the database; System acquisition, including the automatic creation and adjustment of data weights by the acquisition system during the acquisition process.

4. The method for analyzing and adjusting interface data structure based on Huffman tree groups according to claim 2, wherein: Initializing the Huffman tree also includes: Null value processing, including if there is no weight data in the collected system, the data fields and weight data of the collecting system and the collected system are initialized into an empty Huffman tree; The weight data processing includes selecting a node of the weight data as an initialization node and determining a Huffman tree if weight data exists in the acquisition system and the acquired system.

5. The method for analyzing and adjusting interface data structure based on Huffman tree group according to claim 1, characterized in that: The collected system information in the evolution of the Huffman tree structure includes a data field, a collected system name, and request interface information.

6. The method for analyzing and adjusting interface data structure based on Huffman tree group according to claim 1, characterized in that: The alarm prompt method in the unmatched data field alarm processing is one or more of a DingTalk message prompt or an email prompt.

Images