Engineering project data efficient storage system based on cloud computing

Abstract

The invention relates to the technical field of data storage, in particular to an engineering project data efficient storage system based on cloud computing, the system comprises a processor and a memory, and the processor executes a computer program stored in the memory to achieve the following steps: obtaining a combined substring set corresponding to each position span value, and according to the combined sub-string set corresponding to each position span value, obtaining a preference rate corresponding to each position span value, and selecting the position span value corresponding to the maximum preference rate as a target position span value. And finally, according to the frequency of occurrence of different types of combined sub-characters in the combined sub-character string set corresponding to the target position span value, performing Huffman coding on the combined sub-character strings in the combined sub-character string set corresponding to the target position span value, and storing data obtained after coding. And the compression effect of the engineering project information can be improved, so that the high efficiency of engineering project data storage is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to an efficient storage system for engineering project data based on cloud computing. Background Art

[0002] Since engineering project data is relatively important for engineering project management, by collecting and analyzing engineering project data, project managers can monitor the progress of the project in real time to ensure that the project proceeds as planned. In addition, since engineering project data has the characteristics of large data volume, diverse data types, and frequent data updates, Huffman coding is often used to encode and compress engineering project data, and then the compressed data is transmitted and stored for subsequent calls; however, since there may be a large number of repeated fields in the engineering project data, when there are a large number of repeated fields in the engineering project data, it will cause the converted characters to be encoded to have the same frequency of some characters, and this phenomenon will cause more branches in the construction of the Huffman tree, resulting in poor compression effect, and thus the efficient storage of engineering project data cannot be achieved, such as poor storage efficiency. Therefore, when using Huffman coding to compress engineering project data, how to improve the compression effect becomes an urgent problem to be solved. Summary of the invention

[0003] In order to solve the above problems, the present invention provides an efficient storage system for engineering project data based on cloud computing. The technical solutions adopted are as follows: An embodiment of the present invention provides an efficient storage system for engineering project data based on cloud computing, comprising a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the following steps: Get the project string to be encoded; According to different character combination lengths and the frequencies of occurrence of different types of characters in the engineering project character string, the characters in the engineering project character string are combined to obtain combination effect representation values ​​corresponding to various character combination lengths, and the character combination length corresponding to the maximum combination effect representation value is selected as the target character combination length; Obtaining a position span value set according to the frequencies of occurrence of different types of characters in the engineering project character string and the positions of each character in the engineering project character string; According to the target character combination length, each position span value in the position span value set and the project character string, a combination substring set corresponding to each position span value is obtained, and according to the combination substring set corresponding to each position span value, a preference rate corresponding to each position span value is obtained, and the position span value corresponding to the largest preference rate is selected as the target position span value, and according to the frequency of occurrence of different types of combination sub-characters in the combination substring set corresponding to the target position span value, Huffman encoding is performed on the combination substrings in the combination substring set corresponding to the target position span value, and the data obtained after encoding is stored.

[0004] Beneficial effect: The present invention first obtains a project character string to be encoded, and then combines the characters in the project character string according to different character combination lengths and the frequencies of occurrence of different types of characters in the project character string to obtain combination effect representation values ​​corresponding to each character combination length, and selects the character combination length corresponding to the largest combination effect representation value as the target character combination length, and then obtains a position span value set according to the frequencies of occurrence of different types of characters in the project character string and the positions of each character in the project character string, and then obtains a combination substring set corresponding to each position span value according to the target character combination length, each position span value in the position span value set, and the project character string, and obtains a preferred rate corresponding to each position span value according to the combination substring set corresponding to each position span value, and selects the position span value corresponding to the largest preferred rate as the target position span value, and finally performs Huffman coding on the combination substrings in the combination substring set corresponding to the target position span value according to the frequencies of occurrence of different types of combination subcharacters in the combination substring set corresponding to the target position span value, and stores the data obtained after coding. Moreover, the probability that different types of combined substrings in the combined substring set determined by the present invention according to the preferred rate have the same frequency of occurrence is small, thereby being able to improve the compression effect when Huffman coding is used for subsequent compression, that is, the compression rate. That is, this embodiment can improve the compression effect on engineering project information, thereby achieving efficient storage of engineering project data. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0006] Figure 1The present invention is a flowchart of a method for efficiently storing engineering project data based on cloud computing. DETAILED DESCRIPTION

[0007] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the embodiments of the present invention.

[0008] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0009] This embodiment provides an efficient storage system for engineering project data based on cloud computing, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement an efficient storage method for engineering project data based on cloud computing, such as Figure 1 As shown, the efficient storage method for engineering project data based on cloud computing includes the following steps: Step S001, obtaining the engineering project character string to be encoded.

[0010] Since the main purpose of this embodiment is to improve the compression effect of the engineering project data, this embodiment improves the compression effect mainly by adjusting the pre-processing process before compression to improve the subsequent compression effect.

[0011] This embodiment first obtains the engineering project information that needs to be compressed, then converts all Chinese parts in the obtained engineering project information into English, and records the completely converted data as the engineering project character string to be encoded, and when performing the conversion, it can be required to be represented by uppercase and lowercase letters; and in this embodiment, the engineering project information text includes but is not limited to project pre-information, information during construction, and information after project completion, and the engineering project information text that needs to be compressed can be read from a temporary storage file location.

[0012] Therefore, this embodiment obtains the engineering project character string to be encoded through the above process, and in this embodiment, one English letter represents one character.

[0013] Step S002, combining the characters in the engineering project character string according to different character combination lengths and the frequency of occurrence of different types of characters in the engineering project character string, obtaining the combination effect representation value corresponding to each character combination length, and selecting the character combination length corresponding to the largest combination effect representation value as the target character combination length.

[0014] Since the project information contains a large number of repeated fields, after being converted into English, if a single character is used as the basic character to construct the Huffman tree, the compression effect is not good, because the frequency of occurrence of a single character is likely to be the same. Therefore, a combined substring is used as the basic character, and then different lengths of codes are selected according to the different frequencies of occurrence of different types of combined substrings. Similarly, a shorter code is used for a combined substring with a higher frequency of occurrence; and this embodiment subsequently expects that there will be differences in the frequencies of occurrence of different types of combined substrings, so this embodiment needs to first analyze the combination effect representation values ​​corresponding to different character combination lengths, and when the combination effect representation value corresponding to the character combination length is larger, it can be more likely to indicate that when the characters are combined with the character combination length, the frequency difference between different types of combined substrings is more obvious, that is, it is easier to generate the same type of combined substrings; therefore, based on the above description, this embodiment will first combine the characters in the project string according to different character combination lengths and the frequencies of occurrence of different types of characters in the project string, obtain the combination effect representation values ​​corresponding to each character combination length, and select the character combination length corresponding to the largest combination effect representation value as the target character combination length.

[0015] In this embodiment, according to different character combination lengths and the frequencies of occurrence of different types of characters in the project character string, the characters in the project character string are combined to obtain the combination effect representation value corresponding to each character combination length as follows: First, all types of characters appearing in the project string are obtained, and the set constructed by all types of characters appearing in the project string is recorded as a character set. Then, the number of character types appearing in the project string is obtained, and recorded as a type quantity representation value. For example, if the project string is "timeismoney", the character types appearing in the project string are t, i, m, e, s, o, n and y respectively, and the obtained character set is {t, i, m, e, s, o, n, y}, and the type quantity representation value at this time is 8. Then, the frequency of occurrence of each character in the character set in the project string is counted, and recorded as the first frequency value of the corresponding character, and then all the characters in the character set are sorted in descending order according to the first frequency value to obtain a first character sequence. In general, the probability of occurrence of different types of combined substrings generally depends on the character with the lowest frequency. Therefore, when determining the combination effect representation value corresponding to different character combination lengths, it is necessary to determine it according to the result of combining from large to small frequencies, that is, it is necessary to determine it according to the result of combining from large to small frequencies, that is, it is necessary to determine it according to the first character sequence. The characters in the character sequence are combined according to their order; therefore, after obtaining the first character sequence, this embodiment will then obtain a character combination length sequence, and in specific applications, it is necessary to set the minimum character combination length and the maximum character combination length in the character combination length sequence according to actual conditions, but it is required that the character combination lengths in the character combination length sequence are all positive integers. For example, in this embodiment, the minimum character combination length in the character combination length sequence can be 2, and the maximum character combination length can be the upward rounded value of 10% of the total number of characters in the project character string. At this time, the character combination length sequence is composed of all integers between the minimum character combination length and the maximum character combination length; in addition, the character combination length refers to the number of characters in each combination obtained in the subsequent combination. If the character combination length is 2, the number of characters in each combination is 2; then, based on the first character sequence that has been obtained, this embodiment will take the acquisition process of the combination effect representation value corresponding to any character combination length in the character combination length sequence as an example to describe, that is, the acquisition process of the combination effect representation value corresponding to the character combination length is: First, the value of the character combination length is recorded as A1, and A1 different types of characters are randomly selected in the project character string for permutation and combination. Then, after the permutation and combination is completed, the number of combinations obtained is counted and recorded as the permutation and combination value. Then, the reciprocal of the permutation and combination value is obtained and recorded as the first representation value corresponding to the character combination length, and the permutation and combination value can be obtained according to the permutation and combination calculation formula; then, according to the first character sequence and the value A1 of the character combination length, the combination quantity representation value corresponding to the character combination length is obtained, and the result obtained by subtracting the combination quantity representation value from the type quantity representation value is recorded as the feature difference value, and the reciprocal of the result obtained by adding Max(0,D1) to the preset first constant is recorded as the feature ratio value, Max() is the maximum value function, and D1 is the feature difference value; the result obtained by subtracting the feature ratio value from the preset first constant is recorded as the second representation value corresponding to the character combination length; finally, the average of the first representation value and the second representation value is obtained, and recorded as the combination effect representation value corresponding to the character combination length; and the specific calculation expression of the combination effect representation value corresponding to the character combination length is:

[0016] Among them, P0 is the combination effect representation value corresponding to the character combination length, N1 is the number of permutations and combinations, N0 is the number of types, N2 is the number of combinations, and c1 is the preset first constant; and in specific applications, the implementer needs to set the value of the preset first constant according to actual conditions. For example, in this embodiment, the preset first constant can be set to 1; in addition, when N1 and The smaller the value, the larger the P0 value is, and the larger the P0 value is, the more obvious the frequency difference between different types of combined substrings is when characters are combined with the character combination length, that is, it is less likely to produce the same frequency phenomenon between different types of combined substrings.

[0017] In this embodiment, the specific process of obtaining the combination quantity representation value corresponding to the above-mentioned character combination length is as follows: First, determine whether the number of character types in the first character sequence is not less than A1. If so, obtain the combination value corresponding to the first combination type, the second character sequence, and the second frequency value of each character in the second character sequence, and determine whether the number of character types in the second character sequence is not less than A1. If so, obtain the combination value corresponding to the second combination type, the third character sequence, and the third frequency value of each character in the third character sequence, and continue to determine whether the number of character types in the third character sequence is not less than A1. If so, obtain the combination value corresponding to the third combination type, the fourth character sequence, and the fourth frequency value of each character in the fourth character sequence, and continue to determine whether the number of character types in the fourth character sequence is not less than A1. If not, accumulate the combination values ​​of all the obtained combination types, and use the accumulated result as the combination quantity representation value corresponding to the character combination length. As another implementation, the result of accumulating the combination values ​​of all the obtained combination types and adding the special combination number can also be used as the combination quantity representation value corresponding to the character combination length. The special combination number is , is the round-up symbol, A0 is the cumulative sum of the fourth frequency values ​​of all characters in the fourth character sequence; and the first combination type is the combination type formed by the first A1 characters in the first character sequence, and the second combination type is the combination type formed by the first A1 characters in the second character sequence. For example, if a character sequence is {t,i,m,e,o,n,y}, and A1 is 2, then the combination type formed by the first 2 characters in the character sequence is {ti}.

[0018] In addition, in this embodiment, the specific process of obtaining the number of combinations corresponding to the first combination type, the second character sequence, the second frequency value of each character in the second character sequence, the number of combinations corresponding to the second combination type, the third character sequence, the third frequency value of each character in the third character sequence, the number of combinations corresponding to the third combination type, the fourth character sequence, and the fourth frequency value of each character in the fourth character sequence is as follows: The first frequency value of the A1th character in the first character sequence is recorded as the number of combinations corresponding to the first combination type; the first remaining frequency value of each character in the first character sequence is obtained, and the sequence after the characters with the first remaining frequency value of 0 in the first character sequence are removed is recorded as the second character sequence, and the first remaining frequency value of the first A1 characters in the first character sequence is the first quantity difference value of the corresponding characters, the first A1 characters include the A1th character, and the first remaining frequency values ​​of other characters after the A1th character in the first character sequence are the first frequency values ​​of the corresponding characters, the first quantity difference value of the first A1 characters in the first character sequence is the difference between the first frequency value of the corresponding character and the number of combinations corresponding to the first combination type, and the first remaining frequency value of each character in the second character sequence is recorded as the second frequency value of the corresponding character, that is, the first quantity difference value of the a0th character in the first character sequence is the result of subtracting the number of combinations corresponding to the first combination type from the first frequency value of the corresponding character, and the value range of a0 is [1, A1].

[0019] The smallest second frequency value among the first A1 characters in the second character sequence is recorded as the number of combinations corresponding to the second combination type, the second remaining frequency value of each character in the second character sequence is obtained, and the sequence after the characters with the second remaining frequency value of 0 in the second character sequence are eliminated is recorded as the third character sequence, and the second remaining frequency value of the first A1 characters in the second character sequence is the second quantity difference value of the corresponding character, the second remaining frequency value of other characters located after the A1 character in the second character sequence is the second frequency value of the corresponding character, the second quantity difference value of the first A1 characters in the second character sequence is the difference between the second frequency value of the corresponding character and the number of combinations corresponding to the second combination type, and the second remaining frequency value of each character in the third character sequence is recorded as the third frequency value of the corresponding character.

[0020] The specific method for obtaining the number of combinations corresponding to the third combination type, the fourth character sequence, and the fourth frequency value of each character in the fourth character sequence is the same as the specific method for obtaining the number of combinations corresponding to the second combination type, the third character sequence, and the third frequency value of each character in the third character sequence, so this embodiment will not be described in detail.

[0021] And for ease of understanding, this embodiment will illustrate the specific process of obtaining the combination quantity representation value corresponding to the above-mentioned character combination length by example: if the first character sequence at this time is {a, b, c, d}, A1 is 2, and the first frequency value of a is 7, the first frequency value of b is 6, the first frequency value of c is 5, and the first frequency value of d is 1, then the first combination type at this time is {ab}, and according to the first frequency values ​​of a and b, it can be known that the maximum number of combinations belonging to the first combination type {ab} that can be obtained at this time is 6, and after completing the combination of the first combination type {ab}, the first character sequence composed of the remaining characters is {a, c, d}, and at this time the second frequency value of a is 1, the second frequency value of c is 5, and the second frequency value of d is 1, then the second combination type at this time is {ac}, and according to the second frequency values ​​of a and c, it can be known that the maximum number of combinations belonging to the first combination type {ab} that can be obtained at this time is The number of combinations belonging to the second combination type {ac} that can be obtained is 1, and after completing the combination of the second combination type {ac}, the third character sequence composed of the remaining characters is {c, d}, and the third frequency value of c at this time is 4, and the third frequency value of d is 1, so the third combination type at this time is {cd}, and according to the third frequency values ​​of c and d, it can be known that the maximum number of combinations belonging to the third combination type {cd} that can be obtained at this time is 1, and after completing the combination of the third combination type {cd}, only character c remains, then a special combination appears at this time, because the number of c remaining at this time is 3, then the number of special combinations that appear at this time is two, namely the combination {cc} and {c}, and then the above-obtained combination values ​​are accumulated, and the final combination quantity representation value is 6+1+1+2=10.

[0022] Therefore, this embodiment obtains the combination effect representation value corresponding to each character combination length through the above process, and then selects the character combination length corresponding to the largest combination effect representation value as the target character combination length, and the target character combination length can indicate the number of characters in each combined substring finally obtained. If the target character combination length is 2, then the number of characters in each combined substring finally obtained is 2.

[0023] Step S003 , obtaining a position span value set according to the frequencies of occurrence of different types of characters in the engineering project character string and the positions of each character in the engineering project character string.

[0024] Next, this embodiment will obtain a position span value set according to the frequency of occurrence of different types of characters in the project string and the position of each character in the project string; and the purpose of obtaining the position span value set is to make the probability of the different types of combined substrings obtained at last having the same frequency smaller, so the specific process of obtaining the position span value set in this embodiment is: First, the position mark value of each character in the engineering project string is obtained, and the position mark value of the b1th character in the engineering project string is b1; then, in the engineering project string, all characters identical to the first character in the first character sequence are extracted, and a sequence constructed by all the extracted characters is recorded as a first sequence to be analyzed, and all characters identical to the second character in the first character sequence are also extracted, and a sequence constructed by all the extracted characters is recorded as a second sequence to be analyzed, and the characters in the sequence to be analyzed are arranged in ascending order according to the position mark value.

[0025] Next, a mark difference sequence is obtained according to the absolute value of the difference between the position mark values ​​of the characters at the same position in the first sequence to be analyzed and the second sequence to be analyzed, and the d1th mark difference in the mark difference sequence is the absolute value of the difference between the position mark values ​​of the d1th character in the first sequence to be analyzed and the d1th character in the second sequence to be analyzed, and the length of the mark difference sequence is consistent with the length of the shortest sequence to be analyzed between the first sequence to be analyzed and the second sequence to be analyzed.

[0026] Then, the maximum mark difference value in the mark difference sequence is obtained and used as the maximum position span value, the preset first constant is used as the minimum position span value, and the interval formed by the minimum position span value and the maximum position span value is obtained and recorded as the position span interval, and then the set consisting of all integers in the position span interval is recorded as the position span value set.

[0027] Therefore, this embodiment obtains a set of position span values ​​through the above process.

[0028] Step S004, according to the target character combination length, each position span value in the position span value set and the project character string, obtain the combination substring set corresponding to each position span value, and according to the combination substring set corresponding to each position span value, obtain the preference rate corresponding to each position span value, select the position span value corresponding to the largest preference rate as the target position span value, according to the frequency of occurrence of different types of combination sub-characters in the combination substring set corresponding to the target position span value, perform Huffman coding on the combination substrings in the combination substring set corresponding to the target position span value, and store the data obtained after coding.

[0029] After obtaining the position span value set, this embodiment obtains the preference rates of different position span values ​​according to the combined substring set obtained by different position span values, and the greater the preference rate, the smaller the probability that different types of combined substrings in the combined substring set obtained according to the corresponding position span value have the same frequency, and the complexity of reading the combined substring according to the corresponding position span value is lower; therefore, based on the above analysis, it can be seen that this embodiment will then obtain the combined substring set corresponding to each position span value in the position span value set according to the target character combination length, each position span value in the position span value set, and the project character string, and obtain the preference rate corresponding to each position span value according to the combined substring set corresponding to each position span value.

[0030] In this embodiment, according to the target character combination length, each position span value in the position span value set, and the project character string, the specific process of obtaining the combined substring set corresponding to each position span value in the position span value set is as follows: The value of the target character combination length is recorded as A2, and the string formed by the first A2 characters in the first character sequence is recorded as the reference substring; for example, if the first character sequence is {a, b, c, d}, and A2 is 3, then the reference substring is {abc}; then, for ease of understanding, the process of obtaining the combined substring set corresponding to the g0th position span value in the position span interval is described as an example, that is, the process of obtaining the combined substring set corresponding to the g0th position span value is: First, obtain the product of the g0th position span value and A2, and record it as the segmentation length value corresponding to the g0th position span value; then use the segmentation length value corresponding to the g0th position span value to evenly segment the project string without overlapping, and record all substrings obtained by the segmentation as the substrings to be analyzed corresponding to the project string under the g0th position span value; then, according to all the substrings to be analyzed corresponding to the project string under the g0th position span value and the reference substring, obtain the target substring corresponding to each substring to be analyzed, and record the new string formed by the target substrings corresponding to all the substrings to be analyzed as the g0th position span value. The new string corresponding to the 0th position span value; for example, if the project string is {timeonyfirstserved}, and the segmentation length value corresponding to the g0th position span value is 6, then all the substrings to be analyzed corresponding to the project string under the g0th position span value are {timeon}, {yfirst} and {served}, and if the target substring corresponding to {timeon} is W, the target substring corresponding to {yfirst} is E, and the target substring corresponding to {served} is R, then the new string corresponding to the g0th position span value is {WER}.

[0031] Next, according to the g0th position span value and the target character combination length, the characters in the project character string are combined, and the set consisting of all substrings obtained by the combination is recorded as the first set; then, according to the g0th position span value and the target character combination length, the characters in the new character string corresponding to the g0th position span value are combined, and the set consisting of all substrings obtained by the combination is recorded as the second set; then, according to the first set and the second set, the gain representation value corresponding to the g0th position span value is obtained, and it is determined whether the gain representation value corresponding to the g0th position span value is greater than the preset gain threshold value. If so, it indicates that the above-mentioned combination and encoding of the new character string has a better compression effect than the combination and encoding of the project character string, so at this time, the second set is used as the combined substring set corresponding to the g0th position span value; otherwise, it indicates that the above-mentioned combination and encoding of the project character string has a better compression effect than the combination and encoding of the new character string, so at this time, the first set is used as the combined substring set corresponding to the g0th position span value, and the substrings in the combined substring set are all recorded as combined substrings. In addition, in specific applications, the implementer needs to set the preset gain threshold according to actual conditions. For example, in this embodiment, the preset gain threshold can be set to 0.

[0032] In this embodiment, the process of obtaining the target substring corresponding to the substring to be analyzed is as follows: For any substring g1 to be analyzed corresponding to the project string at the g0th position span value: first, all characters in the substring g1 to be analyzed that are the same as the characters in the reference substring are extracted, and the string constructed by all the extracted characters is recorded as a feature string, and the characters in the feature string are arranged in ascending order according to the position mark value; then, according to the feature string, all special character addition positions in the substring g1 to be analyzed are obtained, and special characters are added at all special character addition positions, and the substring g1 to be analyzed after the special characters are added is recorded as the target substring corresponding to the substring g1 to be analyzed; and this step is to combine characters with higher frequency of occurrence as much as possible.

[0033] In this embodiment, the process of obtaining the adding position of the special characters in the substring g1 to be analyzed is as follows: First, all local substrings corresponding to the feature string are obtained, and if the string composed of the j1th character to the j1+A2-1th character in the feature string is the same as the reference substring, then the string composed of the j1th character to the j1+A2-1th character is recorded as the local substring, including the j1th character and the j1+A2-1th character; then, in the substring g1 to be analyzed, the characters with the same position mark values ​​as the characters in the local substring are obtained, and they are all recorded as the characters to be analyzed; and for example, if the substring g1 to be analyzed is {timeonyfirst} and the reference substring is {ti}, then the feature string at this time is {tiit}, and at this time {ti} in the feature string is the same as the reference substring, then the first and second characters in the substring g1{timeonyfirst} to be analyzed are the characters to be analyzed.

[0034] After obtaining the character to be analyzed, the special character adding position in the substring g1 to be analyzed is obtained, and for the k1th character to be analyzed and the k1+1th character to be analyzed in the substring g1 to be analyzed, if the absolute value of the difference between the position mark value of the k1th character to be analyzed and the position mark value between the k1+1th character to be analyzed is less than the g0th position span value, the k1th character to be analyzed is not the last character in the local substring, and the k1+1th character to be analyzed is not the first character in the local substring, then any M0 positions between the k1th character to be analyzed and the k1+1th character to be analyzed are selected as special character adding positions, M0 is the absolute value of the difference between the g0th position span value and the difference representation value, and the difference representation value is the absolute value of the difference between the position mark value of the k1th character to be analyzed and the position mark value of the k1+1th character to be analyzed. For example, if the substring g1 to be analyzed is {timeonyfirst}, the span value of the g0th position is 4, the first and second characters in the substring g1 to be analyzed are the characters to be analyzed, and the string formed by the first two characters in the substring g1 to be analyzed is a local substring, and at this time the absolute value of the difference in position mark values ​​between the first character and the second character in the substring g1 to be analyzed is 1. Since the absolute value of the difference in position mark values ​​between the first character and the second character in the substring g1 to be analyzed is less than 4, the first character in the substring g1 to be analyzed is not the last character in the local substring, and the second character in the substring g1 to be analyzed is not the first character in the local substring, then at this time, 3 special character addition positions need to be randomly added between the first character and the second character in the substring g1 to be analyzed; in addition, it should be noted that the implementer needs to set special characters according to actual conditions. For example, in this embodiment, "#" can be set as a special character.

[0035] In this embodiment, the process of obtaining the first set and the second set is as follows: The project string is recorded as the first string to be read, and starting from the first character on the first string to be read, the characters in the first string to be read are read to obtain the first read substring, and then the characters that have been read in the first string to be read are removed, and the string formed by the remaining characters after the removal is recorded as the second string to be read, and it is determined whether the number of characters in the second string to be read is greater than or not less than , if yes, then starting from the first character in the second string to be read, read the characters in the second string to be read to obtain the second read substring, then remove the characters that have been read in the second string to be read, record the string formed by the remaining characters after the removal as the third string to be read, and determine whether the number of characters in the third string to be read is not less than If not, the third character string to be read is recorded as the third read substring, and the acquisition of the read substring is stopped, and the set constructed by all the acquired read substrings is recorded as the first set, and the absolute value of the difference between the position mark values ​​of the two adjacent characters in the first read substring and the second read substring is the g0th position span value, that is, except for the last read substring acquired, the absolute value of the difference between the position mark values ​​of the two adjacent characters in other read substrings is the g0th position span value, and the lengths of other read substrings except for the last read substring acquired are the target character combination length, and c2 is the preset second constant; in addition, in actual applications, the implementer needs to set the value of c2 according to the actual situation, such as setting the value of c2 to 2.

[0036] Then, the new character string corresponding to the span value of the g0th position is recorded as the first feature string, and because special characters are added to the new character string, the position mark value obtained according to the position of the character in the project string is no longer applicable at this time. Therefore, it is necessary to obtain the feature mark value of each character in the first feature string, and the feature mark value of the v1th character in the first feature string is v1; then starting from the first character on the first feature string, the characters in the first feature string are read to obtain the first feature substring, and then the characters that have been read in the first feature string are removed, and the character string formed by the remaining characters after the removal is recorded as the second feature string, and it is determined whether the number of characters in the second feature string is not less than If yes, then starting from the first character in the second feature string, read the characters in the second feature string to obtain the second feature substring, then remove the characters that have been read in the second feature string, record the string formed by the remaining characters after removal as the third feature string, and determine whether the number of characters in the third feature string is not less than If not, the third feature string is recorded as the third feature substring, and the acquisition of feature substrings is stopped, and the set constructed by all the acquired feature substrings is recorded as the second set, and the absolute value of the difference between the position mark values ​​of two adjacent characters in the first feature substring and the second feature substring is the g0th position span value, that is, except for the last acquired feature substring, the absolute value of the difference between the position mark values ​​of two adjacent characters in other feature substrings is the g0th position span value, and the lengths of other feature substrings except for the last acquired reading substring are all the target character combination lengths.

[0037] For example, if the first string to be read is {timeonyfirst}, the span value of the g0th position is 2, and the target character combination length is 3, then the first substring to be read is {tmo}, the second string to be read is {ienyfirst}, the second substring to be read is {ien}, the third string to be read is {yfirst}, the third substring to be read is {yis}, the fourth string to be read is {frt}, and the last substring to be read is {frt}.

[0038] In this embodiment, the process of obtaining the gain characterization value corresponding to the g0th position span value is as follows: the total number of special characters added in all substrings to be analyzed is counted, and recorded as the character addition quantity value; in the first set, the number of read substrings with an appearance frequency of 1 is obtained, and recorded as the first quantity characteristic value, and the number of read substrings with an appearance frequency of not 1 is obtained, and recorded as the second quantity characteristic value; in the second set, the number of characteristic substrings with an appearance frequency of 1 is obtained, and recorded as the third quantity characteristic value, and the number of characteristic substrings with an appearance frequency of not 1 is obtained, and recorded as the fourth quantity characteristic value; the result obtained by subtracting the first quantity characteristic value from the third quantity characteristic value is obtained, and recorded as the first increase value, and the result obtained by subtracting the second quantity characteristic value from the fourth quantity characteristic value is obtained. The result is recorded as the second increase value, the sum of the first increase value and the number of characters added is recorded as the fifth number characteristic value, Max(0, L1) is recorded as the gain representation value corresponding to the g0th character span value, and L1 is the result obtained by subtracting the fifth number characteristic value from the second increase value; and the larger the gain representation value, the more it indicates that adding special characters can reduce the probability of occurrence of substrings with a frequency of 1, thereby reducing the probability of substrings with different types but the same frequency of occurrence, and two substrings of the same type refer to the two substrings not only having the same length, but also having the same characters in the same positions in the two substrings, such as substrings {yis} and substring {yis} are substrings of the same type, while substrings {yvs} and substring {yis} are substrings of different types.

[0039] Therefore, this embodiment can obtain the combined substring set corresponding to each position span value in the position span value set through the above process, and after obtaining the combined substring set corresponding to each position span value, the preferred rate corresponding to each position span value is obtained according to the combined substring set corresponding to each position span value; and for ease of understanding, the specific acquisition process of the preferred rate corresponding to the g0th position span value in the position span interval is taken as an example for description, that is, the specific acquisition process of the preferred rate corresponding to the g0th position span value is: First, the combined substring set corresponding to the g0th character span value is recorded as the set to be analyzed, and the set constructed by all the combined substring types appearing in the set to be analyzed is recorded as the first type set; then, the frequency of each combined substring type in the first type set appearing in the set to be analyzed is counted, and recorded as the frequency value of the corresponding combined substring type; then, the set constructed by all combined substring types with a frequency value of 1 in the first type set is recorded as the second type set, and the set constructed by all combined substring types in the first type set except the second type set is recorded as the third type set; then, the cumulative sum of the frequency values ​​of all combined substring types in the second type set is obtained, and recorded as the first frequency cumulative value, the cumulative sum of the frequency values ​​of all combined substring types in the third type set is obtained, and recorded as the second frequency cumulative value, and the normalized value of the result obtained by subtracting the second frequency cumulative value from the second frequency cumulative value is recorded as is the first eigenvalue; then the set constructed by the frequency values ​​of all combined substring types in the first type set is recorded as the frequency value set, and the reciprocal of the result obtained by adding the standard deviation of the frequency value set and the preset first constant is recorded as the second ratio, and the result of subtracting the second ratio from the preset first constant is recorded as the second eigenvalue; then the reciprocal of the total number of combined substring types in the second type set is obtained, and recorded as the third eigenvalue; then the reciprocal of the total number of combined substring types in the first type set is obtained, and recorded as the fourth eigenvalue; finally, the average of the first eigenvalue, the second eigenvalue, the third eigenvalue and the fourth eigenvalue is taken as the first preferred value corresponding to the g0th position span value, the normalized value of the ratio of the target character combination length to the segmentation length value corresponding to the g0th position span value is recorded as the second preferred value corresponding to the g0th position span value, and the average of the first preferred value and the second preferred value is recorded as the preferred rate corresponding to the g0th position span value.

[0040] And the calculation expression of the preference rate corresponding to the g0th position span value is:

[0041] in, is the preference rate corresponding to the span value of the g0th position, L1 is the first frequency accumulation value, L2 is the second frequency accumulation value, S1 is the standard deviation of the frequency value set, Q2 is the total number of combined substring types in the second type set, Q1 is the total number of combined substring types in the first type set, Norm() is the normalization function, A2 is the value of the target character combination length, M2 is the segmentation length value corresponding to the span value of the g0th position, and , , is the first preferred value, is the second preferred value; and when and The larger the value, the lower the probability that the frequency values ​​of different types of combined substrings in the set to be analyzed will appear in the same frequency value, and the lower the complexity of group reading, which makes it easier to obtain the combined substrings. The larger the value, the smaller Q2, the larger S1, and the smaller Q1, the smaller the probability that the frequency values ​​of different types of combined substrings in the set to be analyzed appear the same is small. The smaller it is, the smaller the number of substrings to be analyzed corresponding to the project string under the g0th position span value is, so the complexity of obtaining the combined substring is lower, and considering The reason is to avoid the phenomenon of high reading complexity when reading combined substrings.

[0042] Therefore, this embodiment can obtain the preference rate corresponding to each position span value through the above process, and then select the position span value corresponding to the maximum preference rate as the target position span value, and according to the frequency of occurrence of different types of combined sub-characters in the combined sub-character set corresponding to the target position span value, perform Huffman encoding on the combined sub-string in the combined sub-string set corresponding to the target position span value, and store the data obtained after encoding. The greater the frequency of occurrence of the combined sub-character, the greater the length of the encoding. When the frequency of occurrence of different types of combined sub-characters in the combined sub-string set corresponding to the target position span value is known, the process of Huffman encoding is a well-known technology, so it will not be described in this embodiment. In addition, it should be noted that if the compressed data has been restored, the special characters in the restored data need to be removed, and the data after removal needs to be converted into text.

[0043] At this point, this embodiment completes the compressed storage of engineering project information.

[0044] In summary, the present embodiment first obtains the engineering project character string to be encoded, and then combines the characters in the engineering project character string according to different character combination lengths and the frequencies of occurrence of different types of characters in the engineering project character string to obtain the combination effect representation values ​​corresponding to each character combination length, and selects the character combination length corresponding to the maximum combination effect representation value as the target character combination length, and then obtains the position span value set according to the frequencies of occurrence of different types of characters in the engineering project character string and the positions of each character in the engineering project character string, and then obtains the combination substring set corresponding to each position span value according to the target character combination length, each position span value in the position span value set, and the engineering project character string, and obtains the preferred rate corresponding to each position span value according to the combination substring set corresponding to each position span value, and selects the position span value corresponding to the maximum preferred rate as the target position span value, and finally performs Huffman coding on the combination substring in the combination substring set corresponding to the target position span value according to the frequencies of occurrence of different types of combination subcharacters in the combination substring set corresponding to the target position span value, and stores the data obtained after coding. Moreover, the probability that different types of combined substrings in the combined substring set determined according to the preferred rate in this embodiment have the same frequency of occurrence is small, thereby being able to improve the compression effect when Huffman coding is used for subsequent compression, that is, the compression rate. That is, this embodiment can improve the compression effect on engineering project information, thereby achieving efficient storage of engineering project data.

[0045] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A cloud computing-based efficient storage system for engineering project data, comprising a processor and a memory, characterized in that: The processor executes the computer program stored in the memory to implement the following steps: Get the project string to be encoded; Combining the characters in the project character string according to different character combination lengths and the frequencies of occurrence of different types of characters in the project character string to obtain combination effect representation values ​​corresponding to respective character combination lengths, and selecting the character combination length corresponding to the maximum combination effect representation value as the target character combination length; Obtaining a position span value set according to the frequencies of occurrence of different types of characters in the engineering project character string and the positions of each character in the engineering project character string; According to the target character combination length, each position span value in the position span value set, and the project character string, a combination substring set corresponding to each position span value is obtained, and according to the combination substring set corresponding to each position span value, a preference rate corresponding to each position span value is obtained, and the position span value corresponding to the largest preference rate is selected as the target position span value. According to the frequency of occurrence of different types of combination sub-characters in the combination substring set corresponding to the target position span value, Huffman encoding is performed on the combination substrings in the combination substring set corresponding to the target position span value, and the data obtained after encoding is stored.

2. The efficient storage system for engineering project data based on cloud computing according to claim 1, characterized in that: The method for obtaining the combination effect representation value corresponding to each character combination length includes: Recording a set of all types of characters appearing in the project character string as a character set, and recording the number of character types appearing in the project character string as a type quantity representation value; recording the frequency of occurrence of each character in the character set in the project character string as a first frequency value of the corresponding character, and sorting all characters in the character set in descending order of the first frequency values ​​to obtain a first character sequence; For any character combination length: record the value of the character combination length as A1, randomly select A1 different types of characters in the project character string for combination, and record the inverse of the number of combinations obtained after the combination is completed as the first representation value corresponding to the character combination length; according to the first character sequence and the value of the character combination length, obtain the combination quantity representation value corresponding to the character combination length; subtract the combination quantity representation value from the type quantity representation value as the feature difference value, record the inverse of the result obtained by adding Max(0,D1) to the preset first constant as the feature ratio, Max() is the maximum value function, and D1 is the feature difference value; record the result obtained by subtracting the feature ratio from the preset first constant as the second representation value corresponding to the character combination length; record the average of the first representation value and the second representation value as the combination effect representation value corresponding to the character combination length.

3. The efficient storage system for engineering project data based on cloud computing according to claim 2, characterized in that: The method for obtaining the combination quantity representation value corresponding to the character combination length includes: Determine whether the number of character types in the first character sequence is not less than A1. If so, obtain the combination value corresponding to the first combination type, the second character sequence, and the second frequency value of each character in the second character sequence, and determine whether the number of character types in the second character sequence is not less than A1. If so, obtain the combination value corresponding to the second combination type, the third character sequence, and the third frequency value of each character in the third character sequence, and continue to determine whether the number of character types in the third character sequence is not less than A1. If not, accumulate the combination values ​​of all the obtained combination types, and use the accumulated result as the combination quantity representation value corresponding to the character combination length. The first combination type is the combination type composed of the first A1 characters in the first character sequence, and the second combination type is the combination type composed of the first A1 characters in the second character sequence.

4. The efficient storage system for engineering project data based on cloud computing according to claim 3, characterized in that: The method for obtaining the number of combinations corresponding to the first combination type, the second character sequence, the second frequency value of each character in the second character sequence, the number of combinations corresponding to the second combination type, the third character sequence, and the third frequency value of each character in the third character sequence includes: Recording the first frequency value of the A1th character in the first character sequence as the number of combinations corresponding to the first combination type; obtaining the first remaining frequency value of each character in the first character sequence, and recording the sequence after removing characters with first remaining frequency values ​​of 0 in the first character sequence as the second character sequence, the first remaining frequency values ​​of the first A1th character in the first character sequence being the first quantity difference value of the corresponding character, the first remaining frequency values ​​of other characters following the A1th character in the first character sequence being the first frequency value of the corresponding character, and the first quantity difference value of the first A1th character in the first character sequence being the difference between the first frequency value of the corresponding character and the number of combinations corresponding to the first combination type; recording the first remaining frequency value of each character in the second character sequence as the second frequency value of the corresponding character; The minimum second frequency value among the first A1 characters in the second character sequence is recorded as the number of combinations corresponding to the second combination type; the second remaining frequency value of each character in the second character sequence is obtained, and the sequence after removing the characters with the second remaining frequency value of 0 in the second character sequence is recorded as the third character sequence, the second remaining frequency value of the first A1 characters in the second character sequence is the second quantity difference value of the corresponding character, the second remaining frequency value of other characters located after the A1 character in the second character sequence is the second frequency value of the corresponding character, and the second quantity difference value of the first A1 characters in the second character sequence is the difference between the second frequency value of the corresponding character and the number of combinations corresponding to the second combination type; the second remaining frequency value of each character in the third character sequence is recorded as the third frequency value of the corresponding character.

5. The efficient storage system for engineering project data based on cloud computing according to claim 2, characterized in that: The method for obtaining the position span value set includes: Obtaining the position mark value of each character in the engineering project character string, wherein the position mark value of the b1-th character in the engineering project character string is b1; Recording a sequence constructed by all characters in the engineering project character string that are identical to the first character in the first character sequence as a first sequence to be analyzed, and recording a sequence constructed by all characters in the engineering project character string that are identical to the second character in the first character sequence as a second sequence to be analyzed, wherein the characters in the sequences to be analyzed are arranged in ascending order of position mark values; obtaining a marker difference sequence based on the absolute value of the difference between the position marker values ​​of the characters at the same position in the first sequence to be analyzed and the second sequence to be analyzed, wherein the d1th marker difference value in the marker difference sequence is the absolute value of the difference between the position marker values ​​of the d1th character in the first sequence to be analyzed and the d1th character in the second sequence to be analyzed; and using the maximum marker difference value in the marker difference sequence as the maximum position span value; A preset first constant is used as a minimum position span value, an interval formed by the minimum position span value and the maximum position span value is recorded as a position span interval, and a set formed by all integers in the position span interval is recorded as a position span value set.

6. The efficient storage system for engineering project data based on cloud computing according to claim 2, characterized in that: The method for obtaining the combined substring set corresponding to each position span value includes: Record the value of the target character combination length as A2, and record the character string formed by the first A2 characters in the first character sequence as a reference substring; For the g0th position span value in the position span interval: The result of multiplying the g0-th position span value by A2 is recorded as the segmentation length value corresponding to the g0-th position span value; the project character string is evenly segmented without overlapping using the segmentation length value, and all substrings obtained by the segmentation are recorded as the substrings to be analyzed corresponding to the project character string at the g0-th position span value; Obtaining a target substring corresponding to each substring to be analyzed based on all substrings to be analyzed corresponding to the project string at the g0th position span value and the reference substring, and recording a new string formed by the target substrings corresponding to all substrings to be analyzed as a new string corresponding to the g0th position span value; According to the g0th position span value and the target character combination length, the characters in the engineering project character string are combined to obtain a first set; according to the g0th position span value and the target character combination length, the characters in the new character string corresponding to the g0th position span value are combined to obtain a second set; according to the first set and the second set, a gain representation value corresponding to the g0th position span value is obtained, and it is determined whether the gain representation value is greater than a preset gain threshold; if so, the second set is used as the combined substring set corresponding to the g0th position span value; otherwise, the first set is used as the combined substring set corresponding to the g0th position span value.

7. The efficient storage system for engineering project data based on cloud computing according to claim 6, characterized in that: The method for obtaining the target substring corresponding to the substring to be analyzed includes: For any substring g1 to be analyzed corresponding to the project string at the g0th position span value: Recording a character string constructed by all characters in the substring to be analyzed g1 that are identical to characters in the reference substring as a feature character string; obtaining all special character addition positions in the substring to be analyzed g1 based on the feature character string, adding special characters at all special character addition positions, and recording the substring to be analyzed g1 after the special characters are added as a target substring corresponding to the substring to be analyzed g1; The method for obtaining the position where the special characters are added in the substring to be analyzed g1 includes: Get all local substrings corresponding to the feature string. If the string composed of the j1th character to the j1+A2-1th character in the feature string is the same as the reference substring, record the string composed of the j1th character to the j1+A2-1th character as the local substring; record all characters in the substring to be analyzed g1 with the same position mark value as the characters in the local substring as the characters to be analyzed; for the k1th character to be analyzed and the k1+1th character to be analyzed in the substring to be analyzed g1, if the position mark value of the k1th character to be analyzed is the same as the position mark value of the k1+1th character to be analyzed, If the absolute value of the difference between the position mark values ​​of the analyzed characters is less than the g0-th position span value, the k1-th character to be analyzed is not the last character in the local substring, and the k1+1-th character to be analyzed is not the first character in the local substring, then any M0 positions between the k1-th character to be analyzed and the k1+1-th character to be analyzed are added as special character positions, M0 is the absolute value of the difference between the g0-th position span value and the difference representation value, and the difference representation value is the absolute value of the difference between the position mark value of the k1-th character to be analyzed and the position mark value of the k1+1-th character to be analyzed.

8. The efficient storage system for engineering project data based on cloud computing according to claim 7, characterized in that: The method for obtaining the first set and the second set includes: The engineering project character string is recorded as the first character string to be read, and starting from the first character of the first character string to be read, the characters in the first character string to be read are read to obtain the first read substring, and the characters that have been read in the first character string to be read are removed, and the character string formed by the remaining characters after the removal is recorded as the second character string to be read, and it is determined whether the number of characters in the second character string to be read is not less than If so, then starting from the first character in the second string to be read, read the characters in the second string to be read to obtain the second read substring, remove the characters that have been read in the second string to be read, and record the string formed by the remaining characters after the removal as the third string to be read, and determine whether the number of characters in the third string to be read is not less than If not, the third character string to be read is recorded as the third read substring, and the acquisition of the read substring is stopped, and the set constructed by all the acquired read substrings is recorded as the first set, the absolute value of the difference between the position mark values ​​of two adjacent characters in the first read substring and the second read substring is the g0th position span value, and the lengths of the first read substring and the second read substring are both the target character combination lengths; the acquisition method of the second set is the same as the acquisition method of the first set, and c2 is a preset second constant.

9. The efficient storage system for engineering project data based on cloud computing according to claim 8, characterized in that: The method for obtaining the gain representation value corresponding to the g0-th position span value includes: Count the total number of special characters added in all substrings to be analyzed and record it as the number of characters added; In the first set, the number of substrings with an appearance frequency of 1 is recorded as the first quantity characteristic value, and the number of substrings with an appearance frequency of not 1 is recorded as the second quantity characteristic value; in the second set, the number of substrings with an appearance frequency of 1 is recorded as the third quantity characteristic value, and the number of substrings with an appearance frequency of not 1 is recorded as the fourth quantity characteristic value; the result obtained by subtracting the first quantity characteristic value from the third quantity characteristic value is recorded as the first increase value, the result obtained by subtracting the second quantity characteristic value from the fourth quantity characteristic value is recorded as the second increase value, the sum of the first increase value and the character addition quantity value is recorded as the fifth quantity characteristic value, Max(0, L1) is recorded as the gain representation value corresponding to the g0th character span value, and L1 is the result obtained by subtracting the fifth quantity characteristic value from the second increase value.

10. The efficient storage system for engineering project data based on cloud computing according to claim 6, characterized in that: The method for obtaining the preferred ratio corresponding to each position span value includes: For the g0th position span value in the position span interval: record the combined substring set corresponding to the g0th character span value as the set to be analyzed, and record the set constructed by all combined substring types appearing in the set to be analyzed as the first type set; count the frequencies of occurrence of each combined substring type in the first type set in the set to be analyzed, and record them as the frequency values ​​of the corresponding combined substring types; record the set constructed by all combined substring types with a frequency value of 1 in the first type set as the second type set, and record the set constructed by all combined substring types in the first type set except the second type set as the third type set; record the cumulative sum of the frequency values ​​of all combined substring types in the second type set as the first frequency cumulative value, record the cumulative sum of the frequency values ​​of all combined substring types in the third type set as the second frequency cumulative value, and record the second frequency cumulative value. The normalized value of the result obtained by subtracting the first frequency accumulation value from the added value is recorded as the first eigenvalue; the set constructed by the frequency values ​​of all combined substring types in the first type set is recorded as the frequency value set, and the normalized value of the standard deviation of the frequency value set is recorded as the second eigenvalue; the reciprocal of the total number of combined substring types in the second type set is recorded as the third eigenvalue; the reciprocal of the total number of combined substring types in the first type set is recorded as the fourth eigenvalue; the average of the first eigenvalue, the second eigenvalue, the third eigenvalue and the fourth eigenvalue is taken as the first preferred value corresponding to the g0-th position span value; the normalized value of the ratio of the target character combination length to the segmentation length value corresponding to the g0-th position span value is recorded as the second preferred value corresponding to the g0-th position span value, and the average of the first preferred value and the second preferred value is recorded as the preferred rate corresponding to the g0-th position span value.

Images