Compression system for engineering information data

Abstract

The invention relates to the technical field of engineering information data compression processing, in particular to an engineering information data compression system. The system comprises a repeated substring acquisition module for acquiring different types of repeated substrings; the screening substring acquisition module is used for screening the repeated substrings to obtain reference substrings, calculating screening values of the reference substrings and obtaining screening substrings; the optimal grouping length acquisition module is used for dividing the character string to be processed by utilizing each grouping length in the grouping length range so as to obtain an optimal grouping length; the compressed sequence acquisition module is used for segmenting the character string to be processed by utilizing the optimal grouping length to obtain character sub-strings; placing other character sub-strings on the basis of the target character sub-string, and performing displacement adjustment on characters in the character sub-strings to obtain a compressed sequence; and the information sharing module is used for compressing the compressed sequence by using run length coding and transmitting the compressed data to a sharing party. The sharing efficiency of the engineering information can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of engineering information data compression processing, and in particular to a compression system for engineering information data. Background Art

[0002] With the rapid development of the construction industry, the scale and complexity of engineering projects are increasing, and traditional engineering information management methods can no longer meet the needs of modern engineering management. Especially in large-scale engineering projects involving multiple participants, multiple data types and complex business processes, how to effectively share and manage engineering information has become an urgent problem to be solved.

[0003] Since engineering information contains a lot of content and a large amount of repeated text information, run-length encoding is traditionally used for data compression. Run-length encoding is a lossless compression that can ensure the accuracy and consistency of engineering information, but for data with low repeatability, it often increases the amount of data. Although engineering information contains a large amount of repeated content, these repeated contents are relatively scattered. Therefore, directly using run-length encoding for compression will increase the amount of additional data and cannot effectively reduce storage space. In addition, the data packets are too large during transmission, which in turn affects the efficiency of information sharing. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a compression system for engineering information data, and the technical solution adopted is as follows: An embodiment of the present invention provides a system for compressing engineering information data, the system comprising: The repeated substring acquisition module is used to convert the engineering information into a character string to be processed; obtain the occurrence frequency of different characters in the character string to be processed, and obtain different types of repeated substrings according to the occurrence frequency and the interval between characters; A screening substring acquisition module is used to obtain the uniformity of each repeated substring according to the variance of the interval distance between repeated substrings in each repeated substring and the length of each repeated substring; screen the repeated substrings to obtain a reference substring; calculate the screening value of the reference substring according to the uniformity and length of the reference substring, and screen the reference substring according to the screening value to obtain a screening substring; An optimal group length acquisition module is used to obtain a group length range according to the minimum and maximum interval distances between every two screening substrings in each type of screening substring; to divide the character string to be processed using each group length within the group length range, and to obtain an optimal group length based on the division result; The compression sequence acquisition module is used to segment the character string to be processed using the optimal group length to obtain character substrings; take the first character substring appearing in each repeated substring as the target character substring; arrange the target character substrings in order to obtain a reference row; arrange the other character substrings in the columns corresponding to the target character substrings in the reference row according to the existence of repeated substrings in other character substrings, so as to obtain an initial compression sequence; and shift and adjust the characters in each character substring in the initial compression sequence to obtain a compression sequence; The information sharing module is used to compress the compression sequence using run-length coding and transmit the compressed data to the sharing party.

[0005] Preferably, different types of repeated substrings are obtained according to the frequency of occurrence and the interval between characters, including: Sort the characters according to the frequency of occurrence of each character in the character string to be processed to obtain a character sequence, in which there is only one character of each type in the character sequence; record the first character in the character sequence as the first character; obtain the interval distance between every two first characters in the character string to be processed, if the interval distance is greater than or equal to a preset value, select the two first characters with the smallest interval distance and appearing at the front of the character string to be processed, record them as first type characters, form a matching template with the first character of the first type characters and the characters between the two first characters, search for a character substring identical to the matching template in the character string to be processed, obtain a repeated substring of the same type, record it as a first type repeated substring; remove the first type repeated substring in the character string to be processed to obtain a first remaining character string to be processed; Find two first characters that have the same spacing distance as the first type of characters but different spacing characters and are located at the front of the first remaining character string to be processed, and record them as second type of characters; use the first character of the second type of characters and the characters between the two first characters of the second type of characters to form a second matching template, find a character substring that is the same as the matching template in the first remaining character string to be processed, and obtain a repeated substring of the same type, which is recorded as a second type of repeated substring; remove the second type of repeated substring in the first remaining character string to be processed, and obtain a second remaining character string to be processed; When all types of repeated substrings corresponding to the two first characters with the smallest interval distance are found, and so on, all types of repeated substrings corresponding to the two first characters with the second smallest interval distance are found, until all types of repeated substrings corresponding to the first character are found; and so on, according to the order of the characters in the character sequence, different types of repeated substrings corresponding to each character in the character sequence are found.

[0006] Preferably, the calculation formula for uniformity is: , in, Indicates the uniformity of the J-th repeated substring; represents the variance of the interval distance between repeated substrings in the Jth repeated substring; represents the number of J-th repeated substrings; n represents the number of characters in the string to be processed; Indicates the length of the J-th repeated substring; Indicates rounding down.

[0007] Preferably, screening repeated substrings to obtain reference substrings includes: The ratio of the number of each repeated substring to the number of all repeated substrings is the repetition frequency of each repeated substring; the repetition threshold is the inverse of the number of types of repeated substrings. If the repetition frequency of a type of repeated substring is greater than the repetition threshold, the repeated substring of this type is the reference substring.

[0008] Preferably, calculating the screening value of the reference substring according to the uniformity and length of the reference substring, and screening the reference substring according to the screening value to obtain the screening substring comprises: The inverse of the length of the reference substring is obtained, and added to the uniformity of the reference substring to obtain an addition result; the addition result is normalized to obtain a screening value of the reference substring; a screening threshold is set, and a reference substring with a screening value greater than the screening threshold is a screening substring.

[0009] Preferably, the packet length range is obtained according to the minimum value and the maximum value of the interval distance between every two screening substrings in each type of screening substring, including: The minimum interval distance between every two filter substrings in each type of filter substring is obtained respectively, and each minimum interval distance value is added to the preset value to form a minimum interval distance sequence; the maximum interval distance between every two filter substrings in each type of filter substring is obtained respectively, and each maximum interval distance is added to the preset value to form a maximum interval distance sequence; the least common multiple of each value in the minimum interval distance sequence is the lower limit value of the group length range, and the least common multiple of each value in the maximum interval distance sequence is the upper limit value of the group length range.

[0010] Preferably, the character string to be processed is divided using each group length within the group length range, and the optimal group length is obtained based on the division result, including: According to the preset step size, within the group length range, values ​​are taken starting from the lower limit value of the group length range until the upper limit value of the group length range, to obtain different group lengths; the average value of the ratio of the number of reference substrings of each type in the division result corresponding to a group length to the original number of reference substrings of each type is obtained, which is recorded as the division completeness rate corresponding to the group length, and the group length with the largest division completeness rate is selected, which is recorded as the optimal group length.

[0011] Preferably, the target character substrings are arranged in order to obtain a reference row, including: The number of characters of the repeated substring in each target character substring is recorded as the repetition rate of the target character substring; the target character substrings are arranged in descending order of repetition rate to obtain a reference row.

[0012] Preferably, obtaining the initial compression sequence includes: Place the character substrings containing the same repeated substring in the column where the corresponding target character substring is located according to their order in the character string to be processed. After completion, obtain the character substrings that have not been placed to form a set of unplaced substrings; Obtain the similarity between each character substring in the unplaced substring set and each target character substring, where the similarity is the number of identical letters between the character substring and the target character substring, and select each character substring in the unplaced substring set to place in the column where the target character substring with the highest similarity is located, wherein the order in the row direction during placement is determined by the order of each character substring in the unplaced substring set in the character string to be processed; The unplaced character substrings in the unplaced substring set are placed according to their order of appearance in the character string to be processed to obtain an initial compressed sequence.

[0013] Preferably, shifting and adjusting the characters in each character substring in the initial compressed sequence to obtain the compressed sequence comprises: Calculate the similarity between the current character substring and the character substring in the first row or the previous row of the column where the current character substring is located, and select the character substring with the largest similarity from the character substring in the first row or the previous row as the adjustment target; obtain the character string with the same characters in the current character substring and the adjustment target, and record it as the same character string; if the characters in the same position of the same character string corresponding to the current character substring and the same character string corresponding to the adjustment target are the same, the current character substring can be shifted, and if they are not the same, the shift operation cannot be performed; obtain the adjustment efficiency of the character substring that can be shifted according to the number of shift operations when the character substring that can be shifted is shifted and the number of characters in the character substring that can be shifted; if the adjustment efficiency is greater than or equal to the adjustment threshold, perform the shift operation on the character substring that can be shifted; The calculation formula of the adjustment efficiency is: , Among them, δ represents the adjustment efficiency of the character substring that can be shifted; R represents the number of shift operations when the character substring that can be shifted is shifted; max() represents the maximum value operation; Indicates the number of characters in the character substring that can be shifted that overlap with the adjustment target; It indicates the number of characters that overlap with the adjustment target after the character substring that can be shifted is shifted; Q indicates the number of characters in the character substring that can be shifted.

[0014] The embodiments of the present invention have at least the following beneficial effects: the present application converts engineering information into a character string to be processed, and obtains repeated substrings in the character string to be processed; uses the distribution characteristics of repeated substrings in the character string to be processed to analyze and obtain the uniformity of each type of repeated substring, and then screens to obtain reference substrings, and screens the reference substrings again to obtain screened substrings; then analyzes the interval distance between the screened substrings to obtain the most suitable group length, that is, the optimal group length, and then divides the character string to be processed to obtain character substrings, which can ensure that the sequence obtained after such division and adjustment has more continuous repeated characters, thereby improving the compression rate; further, obtains a reference row composed of the target character substring, and then places other character substrings based on the reference row to obtain an initial compression sequence, and then shifts and adjusts the characters in each character substring in the initial compression sequence to obtain a compression sequence, so that the repeatability of the data to be compressed is increased, and the compression principle of run-length encoding can be better adapted, avoiding the additional amount of data, and obtaining less compression space as much as possible without losing information, so as to achieve the purpose of lossless and less compression space occupation, and a higher compression rate, thereby improving the speed of engineering information sharing and increasing the efficiency of engineering information sharing. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A system block diagram of a system for compressing engineering information data provided by an embodiment of the present invention; Figure 2 An example diagram of a possible repeated character string of a compression system for engineering information data provided by an embodiment of the present invention; Figure 3 A schematic diagram of character substrings of a compression system for engineering information data provided by an embodiment of the present invention; Figure 4 A schematic diagram of a reference row of a compression system for engineering information data provided by an embodiment of the present invention; Figure 5 A first sequence schematic diagram of a system for compressing engineering information data provided by an embodiment of the present invention; Figure 6A second sequence schematic diagram of a system for compressing engineering information data provided by an embodiment of the present invention; Figure 7 A schematic diagram of an initial compression sequence of a compression system for engineering information data provided by an embodiment of the present invention; Figure 8 A schematic diagram of a shift operation of a compression system for engineering information data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific scheme of the compression system of engineering information data proposed by the present invention, its specific implementation method, structure, characteristics and effects are described in detail as follows in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0018] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0019] The specific scheme of the system for compressing engineering information data provided by the present invention is described in detail below with reference to the accompanying drawings.

[0020] Example: The main application scenario of the present invention is: when using run-length coding to compress engineering information that needs to be shared, since the text information has multiple repeated fields but the discrete distribution probability is large, the run-length coding compression effect may not achieve the desired effect, and compression cannot be used directly, thereby affecting the efficiency of information sharing and reducing the transmission efficiency when sharing engineering information.

[0021] See also Figure 1 , which shows a block diagram of a compression system for engineering information data provided by an embodiment of the present invention, the system includes the following modules: The repeated substring acquisition module is used to convert the project information into a character string to be processed; obtain the occurrence frequency of different characters in the character string to be processed, and obtain different types of repeated substrings according to the occurrence frequency and the interval between characters.

[0022] The main purpose of this application is to sort and transform the character string converted from the text information according to the distribution of repeated fields in the text information, increase the probability of continuous repetition of characters, so as to better use run-length encoding for compression. First, it is necessary to read the engineering information file to be shared from the database, convert the Chinese part in the engineering information file into English, and use the converted file content as the character string to be processed, wherein after conversion to English, punctuation marks and numbers are also characters, thereby obtaining the character string to be processed. The text information in this application is compressed and transmitted to the sharing party, which can improve the efficiency of data transmission, thereby improving the efficiency of data sharing.

[0023] Since there are repeated contents in the read engineering information, but the repeated contents may not appear continuously, but more often appear scattered, which is not conducive to the use of run-length encoding, it is necessary to sort and adjust the converted character string to be processed. The adjustment principle is to group the character substrings according to the repetitive characteristics, and arrange the substrings in similar combinations as much as possible within a small range. Therefore, it is necessary to first find the repeated substrings in the character string to be processed converted from the engineering information.

[0024] Based on the above analysis, the occurrence frequencies of different characters in the string to be processed are obtained, and different types of repeated substrings are obtained according to the occurrence frequencies and the intervals between characters. Specifically, the process of obtaining each type of repeated substring is as follows: First, count the occurrence frequency of each character in the character string to be processed, and sort the characters from large to small to obtain a character sequence, in which there is only one character of each type in the character sequence; record the first character in the character sequence as the first character; obtain the interval distance between every two first characters in the character string to be processed, if the interval distance is greater than or equal to a preset value, select the two first characters with the smallest interval distance and appearing at the front of the character string to be processed, and record them as first type characters, and use the first character of the first type of characters and the characters between the two first characters to form a matching template, and search for a character substring identical to the matching template in the character string to be processed to obtain a repeated substring of the same type, which is recorded as a first type repeated substring; remove the first type repeated substring in the character string to be processed to obtain a first remaining character string to be processed; Find two first characters that have the same spacing distance as the first type of characters but different spacing characters and are located at the front of the first remaining character string to be processed, and record them as second type of characters; use the first character of the second type of characters and the characters between the two first characters of the second type of characters to form a second matching template, find a character substring that is the same as the matching template in the first remaining character string to be processed, and obtain a repeated substring of the same type, which is recorded as a second type of repeated substring; remove the second type of repeated substring in the first remaining character string to be processed, and obtain a second remaining character string to be processed; When all types of repeated substrings corresponding to the two first characters with the smallest interval distance are found, all types of repeated substrings corresponding to the two first characters with the second smallest interval distance are found, until all types of repeated substrings corresponding to the first characters are found; and so on, according to the order of the characters in the character sequence, different types of repeated substrings corresponding to each character in the character sequence are found.

[0025] For example, the character with the highest frequency is A, that is, the first character is A. Get the spacing distance between A characters in the string to be processed. The spacing distance is the number of characters between two characters. If there is a spacing distance greater than or equal to the preset value 2, then the position of the interval may be the repeated position of the combination of A and the adjacent character. Find the position of A in the same interval, the performance is as follows Figure 2 As shown, Figure 2 The 2 and 6 in the figure represent the spacing between two characters. The small box is the current search character, that is, the first character, and the large box is the possible repeated combinations of the two first characters with the same spacing.

[0026] like Figure 2 , the interval distance between the two first characters is 2, and the characters in the search character when the interval appears for the first time are BC, then the matching template is ABC, and the character substrings identical to the matching template are searched in the string to be processed. These character substrings are repeated substrings of the same type as the matching template, and the successfully matched character substrings are recorded as repeated substrings. , i represents the i-th repeated substring 1. The same is true for the search for the remaining repeated substrings of the same interval of character A. The marked characters are no longer matched, that is, after finding a type of repeated substring, it is necessary to remove it from the character string to be processed, and then use the remaining character string to be processed to find the next type of repeated substring, such as the first remaining character string to be processed and the second remaining character string to be processed after the above removal.

[0027] After character A is found, the above steps are repeated starting from the second most frequent character, that is, the second character in the character sequence, such as character B, until no repeated character substring is found. (J=1, 2, 3…N, i=1, 2, 3…M)}, where J represents the type of repeated substring, i represents the i-th repeated substring of the same type, N represents the number of repeated substring types, and M represents the number of repeated substrings of the same type.

[0028] It should be noted that this method can only find most of the repeated character substrings in the character string to be processed. It cannot ensure that all repeated substrings are found, but most of the repeated substrings are statistically significant.

[0029] The screening substring acquisition module is used to obtain the uniformity of each repeated substring according to the variance of the interval distance between repeated substrings in each repeated substring and the length of each repeated substring; screen the repeated substrings to obtain a reference substring; calculate the screening value of the reference substring according to the uniformity and length of the reference substring, and screen the reference substring according to the screening value to obtain a screening substring.

[0030] The uniformity of the distribution of repeated substrings can reflect the repeated areas in the string, the positions that are more likely to produce repeated changes, and the size of the substrings to a certain extent. Therefore, the uniformity of each repeated substring is obtained according to the variance of the interval distance between repeated substrings in each repeated substring and the length of each repeated substring. Specifically, the calculation formula for the uniformity of repeated substrings is: , in, Indicates the uniformity of the J-th repeated substring; represents the variance of the interval distance between repeated substrings in the Jth repeated substring; represents the number of J-th repeated substrings; n represents the number of characters in the string to be processed; Indicates the length of the J-th repeated substring; Indicates rounding down.

[0031] The smaller the variance, the closer the interval data values ​​of the J-th repeated substring are, that is, the more uniform the distribution of the J-th repeated substring is; It indicates the ratio of the number of J-th repeated substrings to the number of strings after they are divided. The larger the ratio is, the greater the distribution density of the J-th repeated substrings is. The larger the value is, the more uniform the distribution of the J-th repeated substring in the string to be processed is, the greater the distribution density is, and the better the uniformity is.

[0032] The greater the repetition frequency of a repeated substring, the greater the probability that the characters in the string to be processed constitute this type of repeated substring. Therefore, it is necessary to use a repeated substring with a larger repetition probability as a reference substring. When the combination length is short, there are fewer characters to choose from and the total number of possible combinations is also smaller. Therefore, it is easier to have repeated combinations under multiple combinations of the same length. At the same time, if a certain type of reference substring is distributed more evenly in the entire string, it means that this type of repeated substring is more likely to appear in various places in the string to be processed. When splitting the string to be processed, these repeated substrings are less likely to be split. Therefore, based on the above reasons, the screening value of the reference substring can be calculated, and the interval of multiple reference substrings with a screening value greater than the screening threshold is used as the string grouping range. The optimal grouping character substring length is obtained by calculating the complete rate of whether the reference substring is not split after being divided into different groups.

[0033] The repeated substrings are screened to obtain reference substrings. Specifically, the ratio of the number of each repeated substring to the number of all repeated substrings is the repetition frequency of each repeated substring. The repetition threshold is the inverse of the number of types of repeated substrings. If the repetition frequency of a type of repeated substring is greater than the repetition threshold, the repeated substring of this type is the reference substring.

[0034] Furthermore, the screening value of the reference substring is calculated according to the uniformity and length of the reference substring. Specifically, the inverse of the length of the reference substring is obtained and added to the uniformity of the reference substring to obtain the addition result; the addition result is normalized to obtain the screening value of the reference substring. It should be noted that the screening value of each reference substring is the screening value of the reference substring of the type to which the reference substring belongs.

[0035] The calculation formula for the screening value of each type of reference substring is: , where β is the normalized screening value of the selected reference substring, α is the uniformity of the selected reference substring, is the length of the selected reference substring. The larger the screening value, the greater the probability of duplication of the combination of the reference substring and the lower the possibility of being split.

[0036] The reference substring is filtered according to the filtering value to obtain the filtering substring, and a filtering threshold is set. The reference substring with a filtering value greater than the filtering threshold is the filtering substring, thereby filtering the repeated substring to obtain various types of filtering substrings.

[0037] The optimal group length acquisition module is used to obtain the group length range according to the minimum and maximum interval distances between every two filter substrings in each type of filter substring; the character string to be processed is divided using each group length within the group length range, and the optimal group length is obtained based on the division result.

[0038] The filter substring is obtained as described above. It is further necessary to obtain the group length range according to the filter substring so as to determine the most appropriate segmentation length when segmenting the character string to be processed.

[0039] Furthermore, the minimum value of the interval distance between every two filter substrings in each type of filter substring is obtained respectively, and each minimum interval distance value is added to the preset value to form a minimum interval distance sequence; the maximum value of the interval distance between every two filter substrings in each type of filter substring is obtained respectively, and each maximum interval distance is added to the preset value to form a maximum interval distance sequence; the least common multiple of each value in the minimum interval distance sequence is the lower limit value of the group length range, and the least common multiple of each value in the maximum interval distance sequence is the upper limit value of the group length range, thereby obtaining the group length range.

[0040] It should be noted that each type of filter substring will get a minimum interval distance and a maximum interval distance, where the preset value is 1. The reason for adding each minimum interval distance and maximum interval distance to the preset value is that the interval distance does not include the first character of the filter substring.

[0041] Further, according to a preset step length, the preset step length is 1, and within the group length range, values ​​are taken from the lower limit of the group length range according to the preset step length until the upper limit of the group length range, to obtain different group lengths. Use each group length to divide the character string to be processed, obtain the division result corresponding to each group length, and obtain the optimal group length based on the division result. Specifically, obtain the average value of the ratio of the number of reference substrings of each type in the division result corresponding to a group length to the original number of reference substrings of each type, record it as the division completeness rate corresponding to the group length, select the group length with the largest division completeness rate, and record it as the optimal group length. The calculation formula for the division completeness rate of each group length is: , in, represents the division completeness rate corresponding to a group length, K represents the number of reference substring types, It indicates the number of k-th reference substrings after the character string to be processed is divided by the group length; Represents the original number of the k-th reference substring. It indicates the completeness rate of the kth reference substring after grouping with the selected grouping length. The larger the value, the fewer the number of the kth reference substrings that are disassembled after grouping. The larger the value is, the fewer the reference substrings are separated after grouping, and the better the division effect is. Thus, the optimal grouping length can be obtained.

[0042] A compression sequence acquisition module is used to use the optimal grouping length to segment the character string to be processed to obtain character substrings; take the first character substring that appears in each repeated substring as the target character substring; arrange the target character substrings in order to obtain a reference row; arrange the other character substrings in the columns corresponding to each target character substring in the reference row according to the existence of repeated substrings in other character substrings, so as to obtain an initial compression sequence; and shift and adjust the characters in each character substring in the initial compression sequence to obtain a compression sequence.

[0043] After obtaining the optimal group length, the optimal group length is used to segment the character string to be processed to obtain character substrings. Figure 3 As shown, Figure 3The number on each character substring in is the number of the character substring. Since the premise of repetition is that the characters in the character substring are the same and have the same position, but in fact the positions are not the same and the characters may not be the same, it is necessary to first determine some target character substrings, and adjust other character substrings based on the characters and position distribution of the target character substring, so as to increase the number of repeated positions between the character substring and the target character substring as much as possible and the character substrings in the same column can be repeated as much as possible. Therefore, for the remaining character substrings, the target character substring with the highest character similarity should be selected for adjustment. This is because position adjustment can be performed only when there are repeated characters, which increases the probability of overlap between the two character substrings.

[0044] Since the transformation process needs to be recorded when a character substring is adjusted in order to restore the string information during subsequent decompression, and recording this process will take up storage space, it is necessary to analyze and calculate the shift efficiency of each character substring to be adjusted. If the shift consumes too much storage space, that is, the shift efficiency is low, it may lead to unsatisfactory compression effect. Therefore, character substrings with low shift efficiency will not be adjusted to avoid excessive increase in storage content.

[0045] The first character substring appearing in each repeated substring is taken as the target character substring. It should be noted that if two repeated substrings appear in the first character substring, the target character substrings corresponding to the two repeated substrings are both the first character substring, that is, each repeated substring corresponds to a target character substring. It is also possible that two, three or more repeated substrings correspond to one target character substring.

[0046] Then, the target character substrings are sorted. Specifically, the number of characters in the repeated substring in each target character substring is recorded as the repetition rate of the target character substring. The target character substrings are arranged in descending order of repetition rate to obtain a reference row. Figure 4 As shown, the target character substring with the largest repetition rate has the smallest number of columns, and the target character substring with the smallest repetition rate has the largest number of columns.

[0047] Then, according to the existence of repeated substrings in other character substrings, the other character substrings are arranged in the columns corresponding to the target character substrings in the reference row to obtain the initial compression sequence. Specifically, the character substrings containing the same repeated substrings are placed in the columns corresponding to the target character substrings according to their sequential positions in the character string to be processed. After completion, the character substrings that have not been placed are obtained to form a set of unplaced substrings; the sequence obtained after placement is recorded as the first sequence, and the first sequence is as follows Figure 5 shown.

[0048] The similarity between each character substring in the unplaced substring set and each target character substring is obtained, where the similarity is the number of identical letters between the character substring and the target character substring. Each character substring in the unplaced substring set is selected to be placed in the column where the target character substring with the highest similarity is located. The order in the row direction during placement is determined by the order of each character substring in the unplaced substring set in the character string to be processed. The sequence obtained after placement is recorded as the second sequence. The second sequence is as follows: Figure 6 shown.

[0049] Place the unplaced character substrings in the unplaced substring set according to their order of appearance in the string to be processed to obtain the initial compression sequence, such as Figure 7 Then we need to record the position of each character substring to facilitate the subsequent decompression of data, such as Figure 7 As shown in , the record position set is {1, 2, 5, 6, 3, 10, 7, ...}.

[0050] It should be noted that when placing the unplaced character substrings in the unplaced substring set according to their order of appearance in the character string to be processed, it is necessary to obtain the number of character substrings in each column, and then fill the character substrings in each column in ascending order of the number of character substrings in each column, first filling the column with the smallest number of character substrings, then the second smallest, and so on.

[0051] Finally, it is necessary to perform a shift operation on each character substring in the initial compression sequence, calculate the similarity between the current character substring and the character substring in the first row or the previous row of its column, and select the character substring with the largest similarity from the character substring in the first row or the previous row as the adjustment target; obtain the character string with the same characters in the current character substring and the adjustment target, and record it as the same character string; if the characters in the same position of the same character string corresponding to the current character substring and the same character string corresponding to the adjustment target are consistent, then the current character substring can be shifted, if they are not the same, then the shift operation cannot be performed; according to the number of characters whose positions change when the character substring that can be shifted is shifted, the number of shift operations, and the number of characters in the character substring that can be shifted, the adjustment efficiency of the character substring that can be shifted is obtained; if the adjustment efficiency is greater than or equal to the adjustment threshold, the character substring that can be shifted is shifted. The process of the shift operation is as follows: Figure 8 shown.

[0052] The calculation formula for the adjustment efficiency of the character substring that can be shifted is: , Among them, δ represents the adjustment efficiency of the character substring that can be shifted; R represents the number of shift operations when the character substring that can be shifted is shifted; max() represents the maximum value operation; Indicates the number of characters in the character substring that can be shifted that overlap with the adjustment target; It indicates the number of characters that overlap with the adjustment target after the character substring that can be shifted is shifted; Q indicates the number of characters in the character substring that can be shifted.

[0053] It represents the difference between the number of characters overlapped after the shift and the number of shift operations. If the number of characters is greater than the number of shift operations, it means that the shift operation is positive, that is, it reduces memory usage. Otherwise, it means that the shift operation increases additional memory usage. It indicates the effect after shifting, that is, the increase in the overlap rate. The greater the increase in the overlap rate, the better the shift effect. The smaller R is, the fewer times the character substring is shifted. The larger the value is, the more memory usage can be reduced after the shift, the higher the efficiency of character substring adjustment is, and the less storage space the record consumes. The adjustment efficiency of other character substrings can be obtained in the same way.

[0054] For example, the current character substring is ABCD, and the adjustment target is ACBD. The two identical strings are ABC and ACB. Since the characters of the two corresponding identical strings are the same, but the characters at the same position are not exactly the same, ACBD cannot be transformed into ABCD no matter how it is shifted. Therefore, the compressed sequence can be obtained after the shift operation.

[0055] The information sharing module is used to compress the compression sequence using run-length coding and transmit the compressed data to the sharing party.

[0056] Through the above operations, a compressed sequence is obtained. Further, the compressed sequence is read by column, and the read sequence is compressed using run-length encoding to achieve the purpose of compressing the compressed sequence, and a compressed data packet is obtained, and the compressed data packet is transmitted to the receiving end of the sharing personnel. After receiving the compressed data packet, the receiving end decompresses it according to the recorded changes, and outputs the decompressed project file information for the sharing personnel to view.

[0057] In summary, the present invention obtains the character string to be processed converted from the file information, and obtains different types of repeated substrings according to the character frequency and the position interval distribution of the same character. The uniformity of each type of repeated substring is obtained according to the distribution density and position of the same type of repeated substrings, and the reference substring is screened according to the repetition frequency of each type of repeated substring. The screening value of the reference substring is obtained according to the length and uniformity of the reference substring. The group length range of the character sequence is obtained according to the screening value and the highest frequency character distribution interval, and then the optimal group length is obtained, and the character string to be processed is divided to obtain character substrings, and then the character substrings are combined, and the characters in the character substrings are shifted to obtain the compressed sequence after the final adjustment and sorting, and the compressed data packet is obtained after reading by column, and then sent to the sharing personnel. It can improve the efficiency of sharing and reduce storage space.

[0058] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The above is a description of a specific embodiment of this specification. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0059] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A compression system for engineering information data, characterized in that: The system includes: The repeated substring acquisition module is used to convert the engineering information into a character string to be processed; obtain the occurrence frequency of different characters in the character string to be processed, and obtain different types of repeated substrings according to the occurrence frequency and the interval between characters; A screening substring acquisition module is used to obtain the uniformity of each repeated substring according to the variance of the interval distance between repeated substrings in each repeated substring and the length of each repeated substring; screen the repeated substrings to obtain a reference substring; calculate the screening value of the reference substring according to the uniformity and length of the reference substring, and screen the reference substring according to the screening value to obtain a screening substring; An optimal group length acquisition module is used to obtain a group length range according to the minimum and maximum interval distances between every two screening substrings in each type of screening substring; to divide the character string to be processed using each group length within the group length range, and to obtain an optimal group length based on the division result; The compression sequence acquisition module is used to segment the character string to be processed using the optimal group length to obtain character substrings; take the first character substring appearing in each repeated substring as the target character substring; arrange the target character substrings in order to obtain a reference row; arrange the other character substrings in the columns corresponding to the target character substrings in the reference row according to the existence of repeated substrings in other character substrings, so as to obtain an initial compression sequence; and shift and adjust the characters in each character substring in the initial compression sequence to obtain a compression sequence; The information sharing module is used to compress the compression sequence using run-length coding and transmit the compressed data to the sharing party.

2. A system for compressing engineering information data according to claim 1, characterized in that: The method of obtaining different types of repeated substrings according to the occurrence frequency and the interval between characters includes: Sort the characters according to the frequency of occurrence of each character in the character string to be processed to obtain a character sequence, in which there is only one character of each type in the character sequence; record the first character in the character sequence as the first character; obtain the interval distance between every two first characters in the character string to be processed, if the interval distance is greater than or equal to a preset value, select the two first characters with the smallest interval distance and appearing at the front of the character string to be processed, record them as first type characters, form a matching template with the first character of the first type characters and the characters between the two first characters, search for a character substring identical to the matching template in the character string to be processed, obtain a repeated substring of the same type, record it as a first type repeated substring; remove the first type repeated substring in the character string to be processed to obtain a first remaining character string to be processed; Find two first characters that have the same spacing distance as the first type of characters but different spacing characters and are located at the front of the first remaining character string to be processed, and record them as second type of characters; use the first character of the second type of characters and the characters between the two first characters of the second type of characters to form a second matching template, find a character substring that is the same as the matching template in the first remaining character string to be processed, and obtain a repeated substring of the same type, which is recorded as a second type of repeated substring; remove the second type of repeated substring in the first remaining character string to be processed, and obtain a second remaining character string to be processed; When all types of repeated substrings corresponding to the two first characters with the smallest interval distance are found, and so on, all types of repeated substrings corresponding to the two first characters with the second smallest interval distance are found, until all types of repeated substrings corresponding to the first character are found; and so on, according to the order of the characters in the character sequence, different types of repeated substrings corresponding to each character in the character sequence are found.

3. The system for compressing engineering information data according to claim 1, characterized in that: The calculation formula of the uniformity is: ， in, Indicates the uniformity of the J-th repeated substring; represents the variance of the interval distance between repeated substrings in the Jth repeated substring; represents the number of J-th repeated substrings; n represents the number of characters in the string to be processed; Indicates the length of the J-th repeated substring; Indicates rounding down.

4. The system for compressing engineering information data according to claim 1, characterized in that: The step of screening repeated substrings to obtain reference substrings includes: The ratio of the number of each repeated substring to the number of all repeated substrings is the repetition frequency of each repeated substring; the repetition threshold is the inverse of the number of types of repeated substrings. If the repetition frequency of a type of repeated substring is greater than the repetition threshold, the repeated substring of this type is the reference substring.

5. The system for compressing engineering information data according to claim 1, characterized in that: The step of calculating the screening value of the reference substring according to the uniformity and length of the reference substring, and screening the reference substring according to the screening value to obtain the screening substring comprises: The inverse of the length of the reference substring is obtained, and it is added to the uniformity of the reference substring to obtain the addition result; the addition result is normalized to obtain the screening value of the reference substring; a preferred threshold is set, and the reference substring with a screening value greater than the preferred threshold is the screening substring.

6. The system for compressing engineering information data according to claim 1, characterized in that: The obtaining of the packet length range according to the minimum value and the maximum value of the interval distance between every two screening substrings in each type of screening substrings includes: The minimum interval distance between every two filter substrings in each type of filter substring is obtained respectively, and each minimum interval distance value is added to the preset value to form a minimum interval distance sequence; the maximum interval distance between every two filter substrings in each type of filter substring is obtained respectively, and each maximum interval distance is added to the preset value to form a maximum interval distance sequence; the least common multiple of each value in the minimum interval distance sequence is the lower limit value of the group length range, and the least common multiple of each value in the maximum interval distance sequence is the upper limit value of the group length range.

7. The system for compressing engineering information data according to claim 1, characterized in that: The method of dividing the character string to be processed by using each group length within the group length range and obtaining the optimal group length based on the division result includes: According to the preset step size, within the group length range, values ​​are taken starting from the lower limit value of the group length range until the upper limit value of the group length range, to obtain different group lengths; the average value of the ratio of the number of reference substrings of each type in the division result corresponding to a group length to the original number of reference substrings of each type is obtained, which is recorded as the division completeness rate corresponding to the group length, and the group length with the largest division completeness rate is selected, which is recorded as the optimal group length.

8. The system for compressing engineering information data according to claim 1, characterized in that: The step of arranging the target character substrings in order to obtain a reference row includes: The number of characters of the repeated substring in each target character substring is recorded as the repetition rate of the target character substring; the target character substrings are arranged in descending order of repetition rate to obtain a reference row.

9. The system for compressing engineering information data according to claim 1, characterized in that: The obtaining of the initial compression sequence comprises: Place the character substrings containing the same repeated substring in the column where the corresponding target character substring is located according to their order in the character string to be processed. After completion, obtain the character substrings that have not been placed to form a set of unplaced substrings; Obtain the similarity between each character substring in the unplaced substring set and each target character substring, where the similarity is the number of identical letters between the character substring and the target character substring, and select each character substring in the unplaced substring set to place in the column where the target character substring with the highest similarity is located, wherein the order in the row direction during placement is determined by the order of each character substring in the unplaced substring set in the character string to be processed; The unplaced character substrings in the unplaced substring set are placed according to their order of appearance in the character string to be processed to obtain an initial compressed sequence.

10. The system for compressing engineering information data according to claim 1, characterized in that: The step of shifting and adjusting the characters in each character substring in the initial compression sequence to obtain the compression sequence includes: Calculate the similarity between the current character substring and the character substring in the first row or the previous row of the column where the current character substring is located, and select the character substring with the largest similarity from the character substring in the first row or the previous row as the adjustment target; obtain the character string with the same characters in the current character substring and the adjustment target, and record it as the same character string; if the characters in the same position of the same character string corresponding to the current character substring and the same character string corresponding to the adjustment target are the same, the current character substring can be shifted, and if they are not the same, the shift operation cannot be performed; obtain the adjustment efficiency of the character substring that can be shifted according to the number of shift operations when the character substring that can be shifted is shifted and the number of characters in the character substring that can be shifted; if the adjustment efficiency is greater than or equal to the adjustment threshold, perform the shift operation on the character substring that can be shifted; The calculation formula of the adjustment efficiency is: ， Among them, δ represents the adjustment efficiency of the character substring that can be shifted; R represents the number of shift operations when the character substring that can be shifted is shifted; max() represents the maximum value operation; Indicates the number of characters in the character substring that can be shifted that overlap with the adjustment target; It indicates the number of characters that overlap with the adjustment target after the character substring that can be shifted is shifted; Q indicates the number of characters in the character substring that can be shifted.

Images