File consistency detection method and device, electronic equipment and storage medium
By performing data analysis and character splitting hash value calculation on instance files and configuration files, the problem of low accuracy in detecting configuration files and instance files in existing technologies is solved, achieving more efficient and accurate file consistency detection.
Patent Information
- Application Number
- CN202510018436.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-06
- Publication Date
- 2026-07-07
AI Technical Summary
In existing technologies, the accuracy of consistency detection between configuration files and instance files is low, especially the problem of false detection caused by format differences has not been effectively solved.
By analyzing the instance files and configuration files, illegal characters and duplicates are removed. After converting them into a string set, consistency checks are performed using character splitting and hash value calculation to avoid false detections caused by format differences.
It improves the accuracy of file consistency detection, enabling rapid and accurate identification of configuration parameter differences, reducing computational resource consumption, and enhancing the detail and accuracy of detection results.
Smart Images
Figure CN122346459A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, electronic device and storage medium for document consistency detection. Background Technology
[0002] Servers are a crucial component of computer networks. A server includes configuration files and instance files that provide configuration information. The server adjusts its performance and behavior parameters based on this configuration information. Server instance files are generated during server runtime, such as the dynamic variable instance files in MySQL. If the same configuration items in the server's configuration file and instance file are inconsistent, it can lead to server malfunctions. Therefore, consistency checks between the configuration file and instance file are necessary. The results of these checks help developers understand the system's configuration.
[0003] In related technologies, text comparison tools are used to perform consistency checks on instance files and configuration files. These tools typically employ a line-by-line comparison technique. This technique reads and compares the contents of two files line by line to identify differences. It reads each line of both files sequentially; if two lines are identical, it continues to the next line for comparison. If a difference is found, it is recorded, and the comparison of the remaining content is determined as needed. However, this technique is heavily reliant on file format and layout. For example, even if two files are identical in content, formatting differences (such as extra spaces or different line breaks) may lead the line-by-line comparison technique to deem them different. Therefore, the accuracy of consistency checks obtained through line-by-line comparison is relatively low. Improving the accuracy of consistency checks for configuration files and instance files is a pressing issue that needs to be addressed. Summary of the Invention
[0004] This application provides a document consistency detection method, apparatus, electronic device, and storage medium to solve the problem of low accuracy of detection results in the prior art.
[0005] In a first aspect, embodiments of this application provide a document consistency detection method, including:
[0006] Obtain the instance file and configuration file of the target server;
[0007] Data analysis is performed on the instance file and the configuration file respectively to obtain a first string set and a second string set, both of which include multiple strings;
[0008] Perform string matching on the strings in the first string set and the second string set;
[0009] Consistency checks are performed on each matched string pair using character segmentation to obtain a check result, which is used to indicate whether the instance file is consistent with the configuration file.
[0010] In one possible design, obtaining the instance file and configuration file of the target server includes:
[0011] When a change in the state of the target server is detected, a query request is sent to the target server. The query request is used to request the instance files and configuration files of the target server.
[0012] Receive the instance file and the configuration file from the target server;
[0013] The change in the state of the target server includes: the target server switching from a stopped state to a running state or the target server switching from a fault state to a running state.
[0014] In one possible design, the step of performing data analysis on the instance file and the configuration file respectively to obtain a first string set and a second string set includes:
[0015] The instance file and the configuration file are parsed respectively to obtain the dynamic variable dataset and the configuration item dataset; wherein, the dynamic variable dataset includes multiple dynamic variables and the data values of the multiple dynamic variables, and the configuration item dataset includes multiple configuration items and the data values of the multiple configuration items;
[0016] The dynamic variable dataset and the configuration item dataset are preprocessed respectively to obtain the first string set and the second string set; wherein the first string set corresponds to the instance file and the second string set corresponds to the configuration file.
[0017] In one possible design, the preprocessing of the dynamic variable dataset and the configuration item dataset to obtain the first string set and the second string set includes:
[0018] The preprocessed dynamic variable dataset is obtained by performing at least one of the following steps:
[0019] The following methods can be used: First, remove leading and trailing spaces from the dynamic variables in the dynamic variable dataset using string processing functions, as well as leading and trailing spaces from the data values of the dynamic variables; second, identify and remove illegal characters from the dynamic variable dataset using regular expressions; third, merge duplicate dynamic variables in the dynamic variable dataset.
[0020] The data in the preprocessed dynamic variable dataset is converted into strings to obtain the first string set.
[0021] In one possible design, the preprocessing of the dynamic variable dataset and the configuration item dataset to obtain the first string set and the second string set includes:
[0022] The preprocessed configuration item dataset is obtained by performing at least one of the following steps:
[0023] The following methods can be used: First, remove leading and trailing spaces from configuration items in the configuration item dataset using string processing functions, as well as leading and trailing spaces from the data values of configuration items; second, remove annotation data from the configuration item dataset; third, identify and remove illegal characters from the configuration item dataset using regular expressions; and fourth, merge duplicate configuration items in the configuration item dataset.
[0024] The data in the preprocessed configuration item dataset is converted into strings to obtain the second string set.
[0025] In one possible design, the string matching of the strings in the first string set and the second string set includes:
[0026] Iterate through the strings in the first string set, and retrieve the strings from the second string set that match the strings in the first string set, and perform string matching.
[0027] Iterate through the strings in the second string set, and retrieve the strings that match the strings in the second string set from the first string set, and perform string matching.
[0028] In one possible design, the method further includes:
[0029] Unmatched strings in the first string set and the second string set are recorded in the difference data group; the detection result is also used to indicate the difference data group.
[0030] In one possible design, the consistency check performed on each matched string pair using character segmentation to obtain the check result includes:
[0031] Select string A from the first string set, and select string a from the second string set that matches string A; string A and string a are one of the plurality of string pairs;
[0032] Based on a preset number of bytes n, the string A and the string a are split into n substring pairs, denoted as A1 and a1, A2 and a2, ..., An and an; n is a positive integer greater than or equal to 2.
[0033] Obtain the hash value of all substrings in the n substring pairs;
[0034] Based on the hash values of all substrings in the n substring pairs and the weight values corresponding to the n substring pairs, determine the overall consistency value between string A and string a;
[0035] Repeat the above steps until the total consistency value of the multiple string pairs is obtained; determine the detection result based on the total consistency value of the multiple string pairs.
[0036] In one possible design, the method further includes:
[0037] If the total consistency value of the string A and the string a is not 0, then based on the hash values of all substrings in the n substring pairs, determine at least one substring pair with a non-zero consistency value from the n substring pairs.
[0038] Repeat the above steps until all substring pairs with a non-zero consistency value are identified from the plurality of string pairs;
[0039] The detection results are also used to indicate all substring pairs in the plurality of string pairs that have a consistency of 0.
[0040] Secondly, embodiments of this application provide a document consistency detection device, comprising:
[0041] The acquisition module is used to obtain the instance files and configuration files of the target server;
[0042] The processing module is used to perform data analysis on the instance file and the configuration file respectively to obtain a first string set and a second string set, wherein both the first string set and the second string set include multiple strings;
[0043] Perform string matching on the strings in the first string set and the second string set;
[0044] Consistency checks are performed on each matched string pair using character segmentation to obtain a check result, which is used to indicate whether the instance file is consistent with the configuration file.
[0045] In one possible design, the document consistency detection device also includes a sending module and a receiving module.
[0046] The sending module is used to send a query request to the target server when a change in the state of the target server is detected. The query request is used to request the instance file and configuration file of the target server.
[0047] A receiving module is configured to receive the instance file and the configuration file from the target server;
[0048] The change in the state of the target server includes: the target server switching from a stopped state to a running state or the target server switching from a fault state to a running state.
[0049] In one possible design, the processing module is specifically used for:
[0050] The instance file and the configuration file are parsed respectively to obtain the dynamic variable dataset and the configuration item dataset; wherein, the dynamic variable dataset includes multiple dynamic variables and the data values of the multiple dynamic variables, and the configuration item dataset includes multiple configuration items and the data values of the multiple configuration items;
[0051] The dynamic variable dataset and the configuration item dataset are preprocessed respectively to obtain the first string set and the second string set; wherein the first string set corresponds to the instance file and the second string set corresponds to the configuration file.
[0052] In one possible design, the processing module is specifically used for:
[0053] The preprocessed dynamic variable dataset is obtained by performing at least one of the following steps:
[0054] The following methods can be used: First, remove leading and trailing spaces from the dynamic variables in the dynamic variable dataset using string processing functions, as well as leading and trailing spaces from the data values of the dynamic variables; second, identify and remove illegal characters from the dynamic variable dataset using regular expressions; third, merge duplicate dynamic variables in the dynamic variable dataset.
[0055] The data in the preprocessed dynamic variable dataset is converted into strings to obtain the first string set.
[0056] In one possible design, the processing module is specifically used for:
[0057] The preprocessed configuration item dataset is obtained by performing at least one of the following steps:
[0058] The following methods can be used: First, remove leading and trailing spaces from configuration items in the configuration item dataset using string processing functions, as well as leading and trailing spaces from the data values of configuration items; second, remove annotation data from the configuration item dataset; third, identify and remove illegal characters from the configuration item dataset using regular expressions; and fourth, merge duplicate configuration items in the configuration item dataset.
[0059] The data in the preprocessed configuration item dataset is converted into strings to obtain the second string set.
[0060] In one possible design, the processing module is specifically used for:
[0061] Iterate through the strings in the first string set, and retrieve the strings from the second string set that match the strings in the first string set, and perform string matching.
[0062] Iterate through the strings in the second string set, and retrieve the strings that match the strings in the second string set from the first string set, and perform string matching.
[0063] In one possible design, the processing module is specifically used for:
[0064] Unmatched strings in the first string set and the second string set are recorded in the difference data group; the detection result is also used to indicate the difference data group.
[0065] In one possible design, the processing module is specifically used for:
[0066] Select string A from the first string set, and select string a from the second string set that matches string A; string A and string a are one of the plurality of string pairs;
[0067] Based on a preset number of bytes n, the string A and the string a are split into n substring pairs, denoted as A1 and a1, A2 and a2, ..., An and an; n is a positive integer greater than or equal to 2.
[0068] Obtain the hash value of all substrings in the n substring pairs;
[0069] Based on the hash values of all substrings in the n substring pairs and the weight values corresponding to the n substring pairs, determine the overall consistency value between string A and string a.
[0070] Repeat the above steps until the overall consistency value of the multiple string pairs is obtained;
[0071] The detection result is determined based on the overall consistency value of the multiple string pairs.
[0072] In one possible design, the processing module is specifically used for:
[0073] If the total consistency value of the string A and the string a is not 0, then based on the hash values of all substrings in the n substring pairs, determine at least one substring pair with a non-zero consistency value from the n substring pairs.
[0074] Repeat the above steps until all substring pairs with a non-zero consistency value are identified from the plurality of string pairs;
[0075] The detection results are also used to indicate all substring pairs among the plurality of string pairs whose consistency value is not 0.
[0076] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory communicatively connected to the processor;
[0077] The memory stores computer-executed instructions;
[0078] The processor executes computer execution instructions stored in the memory to implement the methods provided by the first aspect and various possible designs.
[0079] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the first aspect and various possible designs.
[0080] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, is used to implement the methods provided in the first aspect and various possible designs.
[0081] This application provides a file consistency detection method, apparatus, electronic device, and storage medium, relating to the field of computer technology. The method includes: acquiring an instance file and a configuration file of a target server; performing data analysis on the instance file and configuration file respectively to obtain a first string set and a second string set, each containing multiple strings; performing string matching on the strings in the first and second string sets; and performing consistency detection on each matched string pair using character segmentation to obtain a detection result, the detection result indicating whether the instance file and the configuration file are consistent. In this technical solution, by performing data analysis on the instance file and configuration file and unifying their format processing, false detections due to format issues are avoided when comparing the instance file and the configuration file. Performing consistency detection on each matched string pair using character segmentation outputs detailed configuration parameter difference value fields in the detection result, improving the accuracy of the detection result. Attached Figure Description
[0082] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0083] Figure 1 A schematic diagram illustrating a scenario for the document consistency detection method provided in this application embodiment;
[0084] Figure 2 A flowchart illustrating an embodiment of the document consistency detection method provided in this application;
[0085] Figure 3 This is a schematic diagram of the string pair consistency detection process provided in an embodiment of this application;
[0086] Figure 4 A flowchart illustrating Embodiment 2 of the document consistency detection method provided in this application;
[0087] Figure 5 This is a schematic diagram of the document consistency detection device provided in the embodiments of this application;
[0088] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0089] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0090] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0091] Before introducing the embodiments of this application, the application background of the embodiments of this application will be explained first:
[0092] Instance files can be viewed as files or collections of data generated during system runtime and associated with a specific instance (such as a database instance, application instance, etc.). In database systems, instance files typically refer to a series of physical files associated with a database instance, containing database data, logs, configuration information, etc. At the application level, instance files may refer to temporary files, log files, configuration files, etc., generated during application runtime. They are crucial for the normal operation of the system or application and require proper management and maintenance.
[0093] Configuration files are an indispensable part of software development, providing programs with flexibility and configurability. By using configuration files appropriately, the adaptability and maintainability of programs can be significantly improved.
[0094] However, during server operation, data loss during system upgrades or transmission can lead to inconsistencies between instance files and configuration files. These inconsistencies increase maintenance complexity and difficulty, and may also create data security risks.
[0095] Commonly used text comparison tools typically perform text comparison line by line. During the comparison process, this line-by-line method may mistakenly identify differences in text format (such as spaces, annotation data, illegal characters, etc.) as differences in text content, resulting in a low accuracy rate for text consistency detection.
[0096] Based on the aforementioned technical problems, the technical concept of this invention is as follows: First, the file consistency detection device periodically detects the target server according to a set time interval. When a change in the target server's state is detected, the device acquires the instance file and configuration file of the target server. Second, the instance file and configuration file undergo data analysis, which includes parsing and preprocessing. The file consistency detection device parses the instance file and configuration file to obtain a dynamic variable dataset and a configuration item dataset. Through preprocessing, it performs operations such as removing illegal characters and merging duplicate items to obtain a first string set and a second string set. The strings in the first string set and the second string set do not include spaces, comment-type data, or illegal characters, facilitating string pair matching between the first string set and the second string set. Then, string matching is performed on the strings in the first string set and the second string set to obtain multiple matched string pairs. Finally, consistency checks are performed on each matched string pair using character segmentation. A matching string is selected from both the first and second string sets to form a string pair. Each string in the pair is then segmented in the same way, and the hash value of each segmented string pair is calculated. Based on the weight value of each string pair, the overall consistency value of the selected strings is calculated. This overall consistency value is then repeatedly calculated for each string pair to obtain the detection result. The detection result indicates whether the instance file and the configuration file are consistent. Performing data analysis on the file before consistency checks avoids misinterpreting format differences (such as spaces, comments, and illegal characters) as content differences during the check, thus improving the accuracy of text consistency checks.
[0097] Figure 1 This is a schematic diagram illustrating a scenario for the file consistency detection method provided in an embodiment of this application. For example... Figure 1 As shown, the scenario includes a file consistency detection device 101 and a target server 102, which are communicatively connected. The file consistency detection device 101 can monitor the operating status of the target server 102 in real time or periodically. When the operating status of the target server 102 changes, the file consistency detection device 101 can obtain instance files and configuration files from the target server 102 and perform file consistency checks on them.
[0098] In this embodiment, the target server can be either a master server or a slave server in a master-slave architecture, and this embodiment does not impose any limitations on this.
[0099] The technical solution of this application will now be described in detail through specific embodiments.
[0100] It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0101] Figure 2 This is a flowchart illustrating a file consistency detection method according to an embodiment of this application. The file consistency detection device exists independently of the target server, and it has a monitoring process for real-time detection of the target server's operating status.
[0102] like Figure 2 As shown, the consistency check method for this file can be implemented through the following steps:
[0103] S21. Obtain the instance file and configuration file of the target server.
[0104] In one possible implementation, S21 can be implemented according to the following S211 and S212:
[0105] S211. When a change in the state of the target server is detected, the detection device sends a query request to the target server. The query request is used to request the instance file and configuration file of the target server.
[0106] In one example, when the detection device detects that the target server's status has changed from a stopped state to a running state, it sends a query request to the target server.
[0107] In another example, when the detection device detects that the target server's status has changed from a faulty state to a running state, it sends a query request to the target server.
[0108] S212. The detection device receives instance files and configuration files from the target server.
[0109] The target server responds to the query request from the detection device, retrieves the instance file and configuration file from the local machine, and sends the instance file and configuration file to the detection device.
[0110] For example, the target server can be a MySQL server, the instance file of the target server can be a dynamic MySQL variable, and the configuration file of the target server can be a static MySQL configuration file.
[0111] S22. Perform data analysis on the instance file and configuration file respectively to obtain the first string set and the second string set. Both the first string set and the second string set include multiple strings.
[0112] In this step, data analysis of the instance files and configuration files includes two steps: parsing and preprocessing. Parsing is the process of reading the instance files and configuration files. A file parser is used to read the instance files and configuration files, obtaining a dynamic variable dataset and a configuration item dataset. The sub-items in the dynamic variable dataset and configuration item dataset are converted into a key-value format of "configuration item: configuration parameter". Preprocessing occurs after parsing the instance files and configuration files, removing illegal characters from the dynamic variable dataset and configuration item dataset, forming a first string set and a second string set. The first string set and the second string set have the same format, both including multiple strings related to configuration items. In one possible implementation of S2, S22 can be implemented based on the following S221 and S222:
[0113] S221. Parse the instance file and configuration file respectively to obtain the dynamic variable dataset and the configuration item dataset.
[0114] By parsing the instance file, a dynamic variable dataset is obtained. Similarly, by parsing the configuration file, a configuration item dataset is obtained. The dynamic variable dataset includes multiple dynamic variables and their values, while the configuration item dataset includes multiple configuration items and their values.
[0115] For example, taking MySQL configuration files and instance files as an example, the MySQL server configuration file is read, and a configuration file parser is used to parse the configuration file, extracting all configuration items and their corresponding data values, which are stored in a configuration item data set as key-value pairs. The configuration file is also parsed using the configuration file parser, extracting all configuration items and their corresponding values, which are also stored in a configuration file value set as key-value pairs, where the key is the configuration item name and the value is the corresponding parameter. The MySQL command-line tool is used to connect to the MySQL instance and obtain the current values of the MySQL instance's dynamic variables; these current values are stored as key-value pairs in a dynamic variable dataset, where the key is the variable name of the current MySQL dynamic variable and the value is the parameter value of the MySQL dynamic variable.
[0116] S222. Preprocess the dynamic variable dataset and the configuration item dataset respectively to obtain the first string set and the second string set.
[0117] In one possible implementation, the dynamic variable dataset is preprocessed to obtain a first string set. The preprocessed dynamic variable dataset is first obtained by performing at least one of the following steps: removing leading and trailing spaces from the dynamic variables and their data values using string processing functions; or identifying and removing illegal characters from the dynamic variable dataset using regular expressions; or merging duplicate dynamic variables in the dataset. The data in the preprocessed dynamic variable dataset is then converted into strings to obtain the first string set.
[0118] In one possible implementation, the configuration item dataset is preprocessed to obtain a second string set. The preprocessed configuration item dataset is first obtained by performing at least one of the following steps: removing leading and trailing spaces from configuration items and data values using string processing functions; or removing annotation-type data from the configuration item dataset; or identifying and removing illegal characters from the configuration item dataset using regular expressions; or merging duplicate configuration items in the configuration item dataset. The data in the preprocessed configuration item dataset is then converted into strings to obtain the second string set.
[0119] For example, a string processing function could be `trim()`, used to remove leading and trailing spaces from strings and variable values. Illegal characters can be comment lines or special characters such as "#". For each line of data, check if it begins with the "#" character. This can be done by checking if the first character of the string is equal to "#", and if a line begins with "#", it can be skipped. For example, ensure comparability between the dynamic variable dataset and the configuration item dataset by unifying their formats, such as the case of parameters, data types, and representations of dates and times. Convert each sub-item in both the dynamic variable dataset and the configuration item dataset into a unified string format, forming a first string set and a second string set.
[0120] S23. Perform string matching on the strings in the first string set and the second string set.
[0121] In one possible implementation, S23 can be implemented according to the following S231 and S232:
[0122] S231. Traverse the strings in the first string set, retrieve the strings that match the strings in the first string set from the second string set, and perform string matching.
[0123] S232. Traverse the strings in the second string set, retrieve the strings from the first string set that match the strings in the second string set, and perform string matching.
[0124] For example, since the sub-items in the first string set and the second string set exist in key-value pairs, if a string in one set has the same configuration name as a string in the other set, then the two strings can be matched.
[0125] In this implementation, by sequentially traversing the first string set and the second string set, multiple matching string pairs can be obtained from the first string set and the second string set, thus speeding up the process.
[0126] In one possible implementation, the process of matching strings in the first and second string sets also includes:
[0127] The strings that are not matched in the first string set and the second string set are recorded in the difference data group. The detection results are also used to indicate the difference data group.
[0128] In this method, server personnel can view the differences in configuration items between the two files by checking the difference data groups in the test results, which facilitates data maintenance on the target server.
[0129] After matching, the first string set and the second string set contain configuration item names that are the same. To confirm whether configuration item parameters with the same name are identical, a consistency check needs to be performed on the first string set and the second string set.
[0130] S24. Perform consistency checks on each matched string pair using character segmentation to obtain the check results. The check results are used to indicate whether the instance file and the configuration file are consistent.
[0131] In one possible implementation, such as Figure 3 As shown, Figure 3 The flowchart for string pair consistency detection provided in this application embodiment is shown below. S24 can be implemented according to S241, S242, S243, S244, S245, and S246:
[0132] S241. Select string A from the first string set, and select string a from the second string set that matches string A.
[0133] For example, a string A from the first string set and a string a from the second string set are selected to form a string pair, in which the configuration item names represented by A and a are the same.
[0134] S242. Based on the preset number of bytes n, split the string A and the string a respectively to obtain n substring pairs, denoted as A1 and a1, A2 and a2, ..., An and an.
[0135] Where n is a positive integer greater than or equal to 2.
[0136] For example, let's take n as 3. Then we get 3 substring pairs, denoted as A1 and a1, A2 and a2, and A3 and a3.
[0137] S243. Obtain the hash value of all substrings in a pair of n substrings.
[0138] For example, when n is 3, the hash value of each string is calculated using hash(X). X represents the string, and hash(X) represents calculating the hash value of the string. Any hash formula can be used to calculate the hash value of the string, such as using the SHA-256 algorithm.
[0139] S244. Based on the hash values of all substrings in the n substring pairs and the weight values corresponding to the n substring pairs, determine the overall consistency value between string A and string a.
[0140] The formula for calculating the overall consistency value is as follows:
[0141]
[0142] Where n represents the preset number of bytes. i is an integer greater than 0 and less than or equal to n, representing the label of the substring after partitioning. The value represents the weight value corresponding to the i-th character pair. This represents calculating the i-th substring of string A using a hash formula. This represents calculating the i-th substring of string a using a hash formula.
[0143] For example, when n is 3, calculate the hash value of the split string pairs in S242, and substitute it into the following formula for calculating the overall consistency value to calculate the overall consistency value of the split string pairs in S242.
[0144]
[0145] S245. Repeat the above steps until the overall consistency value of multiple string pairs is obtained.
[0146] S246. Determine the detection result based on the overall consistency value of multiple string pairs.
[0147] In one example, if the overall consistency value of multiple string pairs is 0, the detection result is used to indicate that the instance file is consistent with the configuration file.
[0148] In another example, if the overall consistency value of at least one string pair among multiple string pairs is not 0, the detection result is used to indicate that the instance file is inconsistent with the configuration file.
[0149] In the above steps, strings A and a are split and their hash values are calculated separately. Compared to line-by-line comparison, this reduces the computational space required during the detection process and saves system resources. Furthermore, by calculating the overall consistency value of the string pairs, this consistency value indicates whether the instance file and the configuration file are consistent. Compared to line-by-line comparison, this method yields the detection results in a shorter time.
[0150] The file consistency detection method provided in this application periodically monitors the status of the target server to determine the opportune time to obtain the instance files and configuration files of the target server. Simultaneously, it performs data analysis on the instance files and configuration files, removing illegal characters and merging duplicates through parsing and data preprocessing to obtain a dynamic variable dataset and a configuration item dataset. These datasets are then converted into string formats, resulting in a first string set and a second string set. This data processing improves file standardization, avoids false positives due to formatting issues, and enhances the accuracy of the detection results. String matching is performed on the strings in the first and second string sets. The overall consistency value is calculated for each string pair using string segmentation, resulting in a shorter detection time compared to line-by-line comparison techniques.
[0151] In the aforementioned embodiments, during the consistency detection of each matched string pair using character segmentation, there are cases where the total consistency value of some string pairs is not 0. For these differing string pairs, the difference position of the string pairs can be further determined, and the detection result can include the name of the difference configuration item and the difference parameter value.
[0152] Figure 4 This is a flowchart illustrating Embodiment 2 of the document consistency detection method provided in this application. Figure 4 As shown, the file consistency detection method also includes:
[0153] S41. Based on the hash values of all substrings in the n substring pairs, determine at least one substring pair from the n substring pairs whose consistency is not 0.
[0154] For example, let n=3. When one string pair is different, such as... This indicates that strings A1 and a1 are not the same. When two strings are different, such as...
[0155] This indicates that strings A1 and a1, and A2 and a1 are not identical. When three strings are different, such as...
[0156] If not, it means that strings A1 and a1, A2 and a1, and A3 and a3 are inconsistent. Record the inconsistent strings that occur during the comparison.
[0157] S42 repeats the above steps until all substring pairs with non-zero consistency are identified from multiple string pairs.
[0158] The detection results are also used to indicate all substring pairs in multiple string pairs that have a consistency value of 0.
[0159] In the above steps, during the consistency check for each matched string pair, there are some string pairs where the overall consistency value is not 0. For these non-zero consistency string pairs, for one of them, the hash values of the n partitions of its substring pairs are compared. The difference string segments are determined based on the results of the different hash value comparisons. Compared to line-by-line comparison techniques, this method can detect difference segments faster. Furthermore, because the detection results indicate the difference string segments, the results are more accurate compared to line-by-line comparison techniques.
[0160] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.
[0161] Figure 5 This is a schematic diagram of the document consistency detection device provided in an embodiment of this application. Figure 5 As shown, the file consistency detection device 50 is applied to the target server, and the file consistency detection device 50 includes:
[0162] This application provides a document consistency detection device, including:
[0163] Module 51 is used to obtain the instance file and configuration file of the target server;
[0164] Processing module 52 is used to perform data analysis on instance files and configuration files respectively to obtain a first string set and a second string set, both of which include multiple strings;
[0165] Perform string matching on the strings in the first string set and the second string set;
[0166] Consistency checks are performed on each matched string pair using character segmentation to obtain the results, which are used to indicate whether the instance file and the configuration file are consistent.
[0167] In one possible design, the document consistency detection device also includes a sending module 53 and a receiving module 54.
[0168] The sending module 53 is used to send a query request to the target server when a change in the state of the target server is detected. The query request is used to request the instance file and configuration file of the target server.
[0169] The receiving module 54 is used to receive instance files and configuration files from the target server;
[0170] The target server's state changes, including: the target server switching from a stopped state to a running state or the target server switching from a faulty state to a running state.
[0171] In one possible design, processing module 52 is specifically used for:
[0172] The instance file and the configuration file are parsed respectively to obtain the dynamic variable dataset and the configuration item dataset; wherein, the dynamic variable dataset includes multiple dynamic variables and multiple dynamic variable data values, and the configuration item dataset includes multiple configuration items and multiple configuration item data values;
[0173] The dynamic variable dataset and configuration item dataset are preprocessed separately to obtain a first string set and a second string set; the first string set corresponds to the instance file, and the second string set corresponds to the configuration file.
[0174] In one possible design, processing module 52 is specifically used for:
[0175] The preprocessed dynamic variable dataset is obtained by performing at least one of the following steps:
[0176] Use string processing functions to remove leading and trailing spaces from dynamic variables in a dynamic variable dataset, as well as leading and trailing spaces from the data values of dynamic variables; or use regular expressions to identify and remove illegal characters from a dynamic variable dataset; or merge duplicate dynamic variables in a dynamic variable dataset.
[0177] The data in the preprocessed dynamic variable dataset is converted into strings to obtain the first string set.
[0178] In one possible design, processing module 52 is specifically used for:
[0179] The preprocessed configuration item dataset is obtained by performing at least one of the following steps:
[0180] Use string processing functions to remove leading and trailing spaces from configuration items in the configuration item dataset, as well as leading and trailing spaces from the data values of configuration items; or remove comment data from the configuration item dataset; or identify and remove illegal characters from the configuration item dataset using regular expressions; or merge duplicate configuration items in the configuration item dataset.
[0181] The data in the preprocessed configuration item dataset is converted into strings to obtain the second string set.
[0182] In one possible design, processing module 52 is specifically used for:
[0183] Iterate through the strings in the first string set, and retrieve the strings in the second string set that match the strings in the first string set, and perform string matching.
[0184] Iterate through the strings in the second string set, retrieve the strings from the first string set that match the strings in the second string set, and perform string matching.
[0185] In one possible design, processing module 52 is specifically used for:
[0186] Unmatched strings in the first and second string sets are recorded in the difference data group; the detection results are also used to indicate the difference data group.
[0187] In one possible design, processing module 52 is specifically used for:
[0188] Select string A from the first string set, and select string a from the second string set that matches string A; string A and string a are one of multiple string pairs;
[0189] Based on the preset number of bytes n, string A and string a are split into n substring pairs, denoted as A1 and a1, A2 and a2, ..., An and an; n is a positive integer greater than or equal to 2;
[0190] Get the hash value of all substrings in a pair of n substrings;
[0191] Based on the hash values of all substrings in the n substring pairs and the corresponding weight values of the n substring pairs, determine the overall consistency value between string A and string a.
[0192] Repeat the above steps until a total consistency value for multiple string pairs is obtained;
[0193] The detection result is determined based on the overall consistency value of multiple string pairs.
[0194] In one possible design, processing module 52 is specifically used for:
[0195] If the total consistency value of string A and string a is not 0, determine at least one substring pair with a non-zero consistency value from the n substring pairs based on the hash values of all substrings in the n substring pairs;
[0196] Repeat the above steps until all substring pairs with a non-zero consistency value are identified from multiple string pairs;
[0197] The test results are also used to indicate all substring pairs in multiple string pairs that have a consistency value that is not 0.
[0198] This application provides a file consistency detection device, which can be used to execute the file consistency detection method in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.
[0199] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls, or entirely in hardware. Alternatively, some modules can be implemented through processing element calls in software, while others are implemented in hardware. Moreover, these modules can be fully or partially integrated together, or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions.
[0200] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 60 includes a processor 61, a memory 62, and computer program instructions stored in the memory 62 and executable on the processor 61. When the processor 61 executes the computer program instructions, it implements the file consistency detection method provided in any of the foregoing embodiments.
[0201] Optionally, the various devices on the electronic device can be connected to each other via a system bus.
[0202] The memory 62 can be a separate memory unit or a memory unit integrated into the processor. The number of processors can be one or more.
[0203] Optionally, the electronic device may also include a communication interface for interacting with other devices.
[0204] It should be understood that processor 61 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0205] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.
[0206] All or part of the steps in the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above-described method embodiments. The aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.
[0207] The electronic device provided in this application embodiment can be used to execute the file consistency detection method provided in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.
[0208] This application provides a computer-readable storage medium storing computer-executable instructions. When these instructions are executed on a computer, the computer performs the aforementioned file consistency detection method.
[0209] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0210] Optionally, a readable storage medium can be coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Alternatively, the readable storage medium can be an integral part of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.
[0211] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the above-described file consistency detection and control method.
[0212] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A method of detecting file consistency, characterized by, include: Obtain the instance file and configuration file of the target server; Data analysis is performed on the instance file and the configuration file respectively to obtain a first string set and a second string set, both of which include multiple strings; Perform string matching on the strings in the first string set and the second string set; Consistency checks are performed on each matched string pair using character segmentation to obtain a check result, which is used to indicate whether the instance file is consistent with the configuration file.
2. The method of claim 1, wherein, The process of obtaining the instance file and configuration file of the target server includes: When a change in the state of the target server is detected, a query request is sent to the target server. The query request is used to request the instance files and configuration files of the target server. Receive the instance file and the configuration file from the target server; The change in the state of the target server includes: the target server switching from a stopped state to a running state, or the target server switching from a fault state to a running state.
3. The method according to claim 1 or 2, characterized in that, The step of performing data analysis on the instance file and the configuration file respectively to obtain a first string set and a second string set includes: The instance file and the configuration file are parsed respectively to obtain the dynamic variable dataset and the configuration item dataset; wherein, the dynamic variable dataset includes multiple dynamic variables and the data values of the multiple dynamic variables, and the configuration item dataset includes multiple configuration items and the data values of the multiple configuration items; The dynamic variable dataset and the configuration item dataset are preprocessed respectively to obtain the first string set and the second string set; wherein the first string set corresponds to the instance file and the second string set corresponds to the configuration file.
4. The method according to claim 3, characterized in that, The step of preprocessing the dynamic variable dataset and the configuration item dataset to obtain the first string set and the second string set includes: The preprocessed dynamic variable dataset is obtained by performing at least one of the following steps: The following methods can be used: First, remove leading and trailing spaces from the dynamic variables in the dynamic variable dataset using string processing functions, as well as leading and trailing spaces from the data values of the dynamic variables; second, identify and remove illegal characters from the dynamic variable dataset using regular expressions; third, merge duplicate dynamic variables in the dynamic variable dataset. The data in the preprocessed dynamic variable dataset is converted into strings to obtain the first string set.
5. The method according to claim 3, characterized in that, The step of preprocessing the dynamic variable dataset and the configuration item dataset to obtain the first string set and the second string set includes: The preprocessed configuration item dataset is obtained by performing at least one of the following steps: The following methods can be used: First, remove leading and trailing spaces from configuration items in the configuration item dataset using string processing functions, as well as leading and trailing spaces from the data values of configuration items; second, remove annotation data from the configuration item dataset; third, identify and remove illegal characters from the configuration item dataset using regular expressions; and fourth, merge duplicate configuration items in the configuration item dataset. The data in the preprocessed configuration item dataset is converted into strings to obtain the second string set.
6. The method according to any one of claims 1 to 5, characterized in that, The string matching process for the strings in the first string set and the second string set includes: Iterate through the strings in the first string set, and retrieve the strings from the second string set that match the strings in the first string set, and perform string matching. Iterate through the strings in the second string set, and retrieve the strings that match the strings in the second string set from the first string set, and perform string matching.
7. The method according to claim 6, characterized in that, The method further includes: Unmatched strings in the first string set and the second string set are recorded in the difference data group; the detection result is also used to indicate the difference data group.
8. The method according to any one of claims 1 to 7, characterized in that, The consistency check is performed on each matched string pair using character segmentation to obtain the check result, including: Select string A from the first string set, and select string a from the second string set that matches string A; string A and string a are one of the plurality of string pairs; Based on a preset number of bytes n, the string A and the string a are split into n substring pairs, denoted as A1 and a1, A2 and a2, ..., An and an; n is a positive integer greater than or equal to 2. Obtain the hash value of all substrings in the n substring pairs; Based on the hash values of all substrings in the n substring pairs and the weight values corresponding to the n substring pairs, determine the overall consistency value between string A and string a; Repeat the above steps until the overall consistency value of the multiple string pairs is obtained; The detection result is determined based on the overall consistency value of the multiple string pairs.
9. The method according to claim 8, characterized in that, The method further includes: If the total consistency value of the string A and the string a is not 0, then based on the hash values of all substrings in the n substring pairs, determine at least one substring pair with a non-zero consistency value from the n substring pairs. Repeat the above steps until all substring pairs with a non-zero consistency value are identified from the plurality of string pairs; The detection results are also used to indicate all substring pairs in the plurality of string pairs that have a consistency of 0.
10. A document consistency detection device, characterized in that, include: The acquisition module is used to obtain the instance files and configuration files of the target server; The processing module is used to perform data analysis on the instance file and the configuration file respectively to obtain a first string set and a second string set, wherein both the first string set and the second string set include multiple strings; Perform string matching on the strings in the first string set and the second string set; Consistency checks are performed on each matched string pair using character segmentation to obtain a check result, which is used to indicate whether the instance file is consistent with the configuration file.
11. An electronic device, characterized in that, include: Processor; memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 9.
13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 9.