Chinese character grouping test method, system and computer readable storage medium based on word balance

Abstract

The present application belongs to the technical field of information security, and particularly relates to a Chinese character grouping test method and system based on word balance and a computer readable storage medium. The method comprises the following steps: S1, obtaining a Chinese text dataset, performing word segmentation processing, counting and sorting word frequency and word frequency, and obtaining a word frequency table and a word frequency table; S2, calculating the number of groups to be grouped; S3, grouping Chinese characters according to word frequency, and excluding high-frequency words; the high-frequency words are words with a word frequency greater than a set threshold; S4, setting a sliding window size and a sliding distance, traversing the Chinese text dataset obtained in step S1, and calculating the average embedding rate of the groups. The present application has the characteristics that Chinese characters can be grouped based on word balance, the sliding window method is used to test the grouping embedding rate, and the embedding rate of a tracing system is effectively improved under the condition that the carrier contains a small number of characters.

Description

Technical Field

[0001] The present invention belongs to the technical field of information security, and particularly relates to a Chinese character grouping test method, system and computer-readable storage medium based on word and character balance. Background Technique

[0002] The Chinese character glyph watermark algorithm is a watermark algorithm that uses Chinese character glyphs to hide "0" or "1". This algorithm aims to equip the same Chinese character with multiple different glyphs and encodings. These glyphs are generated by slightly adjusting the strokes. In different electronic documents, different glyphs are used to hide watermark information. This watermark algorithm does not change the overall appearance of the text and has strong invisibility. Even after the carrier is photographed or screenshot, the encoding of the character can still be recognized. It has the advantages of strong anti-attack ability and high robustness. This method is usually applied to traceability tasks.

[0003] In existing traceability tasks, an identity sequence code is generated based on the MAC address, IP address, host name, user name, and current time of the host, and the identity sequence code is converted into a binary bit sequence. A set of deformed characters is generated from commonly used Chinese characters, grouped in descending order of word frequency, and the binary bit sequence is embedded into the Chinese characters in each group bit by bit. However, there are still the following problems: Since the frequencies of high-frequency characters vary greatly, sequential grouping easily leads to uneven word frequency distribution; and high-frequency words are not excluded during grouping. Due to the high occurrence frequency of high-frequency words, if they appear in different groups, the information of two groups can be extracted. On the contrary, if they are in the same group, only the information of one group can be extracted, resulting in a decrease in the embedding rate. In existing solutions, the same number of variants, such as 4 kinds, containing 2-bit information, are usually generated for the selected Chinese characters, which are called 2-bit characters. However, for some high-frequency characters, such as the character "大" with a relatively simple structure, at most only 2 variants can be generated, containing 1-bit information, which are called 1-bit characters. There is currently no good application solution for such characters.

[0004] Therefore, it is very important to design a Chinese character grouping test method, system and computer-readable storage medium based on word and character balance that can group Chinese characters based on word and character balance, use the sliding window method to test the grouping embedding rate, and effectively improve the embedding rate of the traceability system when the number of characters contained in the carrier is small.

[0005] For example, Chinese patent document CN201911259549.1 describes a watermark embedding method, watermark tracing method, and apparatus. This method obtains a digital watermark sequence to be embedded based on a digital watermark sequence and a preset prime number sequence; it uses a cryptographic storage algorithm to segment the sequence, obtaining a preset number of sub-digital watermarks to be embedded; and it uses a preset grouping algorithm to group the data to be processed according to the data identifier, obtaining a preset number of data groups to be processed and their corresponding identifiers. Then, using preset embedding rules, it embeds one sub-digital watermark into the data of one data group to be processed, resulting in data with embedded digital watermarks. In the watermark tracing process, the watermark information is extracted from the data with embedded digital watermarks by reversing the watermark embedding process. The extracted watermark information is compared with the original watermark information to achieve data tracing. Although this reduces the complexity of tracing and improves its accuracy, its drawback is that uneven grouping leads to incomplete embedding groups, ultimately causing missing groups during extraction. Summary of the Invention

[0006] This invention aims to overcome the problem in existing traceability systems where uneven grouping leads to incomplete embedding and ultimately missing groups during extraction. It provides a word-balanced Chinese character grouping testing method, system, and computer-readable storage medium that can group Chinese characters based on word balance, use a sliding window method to test the group embedding rate, and effectively improve the embedding rate of the traceability system even when the carrier contains a small number of characters.

[0007] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0008] The Chinese character grouping test method based on word balance includes the following steps:

[0009] S1. Obtain the Chinese text dataset, perform word segmentation, count and sort the character and word frequencies, and obtain the character frequency table and word frequency table;

[0010] S2, calculate the number of groups that need to be formed;

[0011] S3, group Chinese characters according to their frequency and exclude high-frequency words; the high-frequency words are those whose frequency is greater than a set threshold.

[0012] S4. Set the sliding window size and sliding distance, traverse the Chinese text dataset obtained in step S1, and calculate the average embedding rate of the groups.

[0013] Preferably, step S1 includes the following steps:

[0014] S11: Traverse the Chinese text dataset, count the frequency of Chinese characters and sort them;

[0015] S12 uses a word segmentation tool to segment the Chinese text dataset, counts the word frequency of Chinese characters, and sorts them.

[0016] Preferably, step S2 includes the following steps:

[0017] S21. Generate an identity sequence code based on the host's MAC address, IP address, hostname, username, and current time, and convert it into a binary bit sequence with a length of length.

[0018] S22, calculate k = length / 2, where k is the number of groups when only 2-bit words are used;

[0019] S23, will Divide into m groups. Divided into 2 groups, so The group only needs to be divided into k-1 groups, so m=k-1;

[0020] The input Chinese character encoding file contains two types of variant characters: 2-bit characters and 1-bit characters, labeled as follows: and .

[0021] Preferably, step S3 includes the following steps:

[0022] S31, Calculation , The sum of the frequencies of all characters in the Chinese character set and The calculated total frequency of each group of characters is as follows: and And set the error range value. ;

[0023] S32, sequential traversal of the set Chinese characters are grouped to ensure even distribution of character frequencies, and high-frequency words within the same group are eliminated by traversing the word frequency table.

[0024] S33, sequential traversal of the set Chinese characters are grouped to ensure even frequency distribution, and high-frequency words within the same group are eliminated by traversing the word frequency table. The number of groups is fixed at 2.

[0025] During the initialization of the character frequency table, each Chinese character is assigned an initial value. .

[0026] Preferably, step S4 includes the following steps:

[0027] S41, set the window size to e and the sliding distance to g. First, extract the string with coordinates [0, e-1]. If the string contains at least one character in each group, it is successfully embedded and counted. Then, slide and extract the string with coordinates [g, e+g-1] to determine whether the embedding is successful.

[0028] S42, traverse the Chinese text dataset obtained in step S1 according to step S41;

[0029] S43. Divide the number of successful embeddings by the number of judgments to obtain the average embedding rate.

[0030] This invention also provides a Chinese character grouping test system based on word balance, including:

[0031] The list traversal module is used to retrieve characters from a list of variant characters;

[0032] The grouped storage module is used to store the words retrieved by the list traversal module;

[0033] The vocabulary judgment module is used to determine whether the characters taken by the list traversal module and the characters in the group storage module can form a word;

[0034] The threshold judgment module is used to determine whether the sum of word frequencies in the group storage module reaches the threshold.

[0035] As a preferred option, the system also includes:

[0036] The grouping judgment module is used to determine whether the input group contains all the variant characters.

[0037] As a preferred option, the system also includes:

[0038] The embedding rate test module is used to calculate the average embedding rate of groups.

[0039] The present invention also provides a computer-readable storage medium including computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of the Chinese character grouping test method based on word balance described in any of the above embodiments.

[0040] Compared with the prior art, the beneficial effects of this invention are: (1) This invention statistically analyzes a large number of Chinese datasets and calculates the word frequency. By evenly distributing the word frequency in each group, high-frequency words and low-frequency words can complement each other. The dataset is segmented into words to analyze the word frequency in the data set and avoid the occurrence of high-frequency words in the same group when grouping. (2) This invention also divides high-frequency words with simple structures into two separate groups, which carry a portion of bit information and improve the overall grouping frequency. (3) This invention also provides a method for calculating the average embedding rate. By comparing the method proposed in this invention with existing methods, a better embedding rate result is obtained. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating Chinese character grouping as provided in an embodiment of the present invention. Figure 2 This is a flowchart of a group embedding rate test provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a grouping judgment module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an embedding rate testing module provided in an embodiment of the present invention. Detailed Implementation

[0042] To more clearly illustrate the embodiments of the present invention, specific implementation methods will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0043] Example 1:

[0044] This invention provides a Chinese character grouping test method based on word balance, including a Chinese character grouping process and a group embedding rate test process.

[0045] S1, grouping Chinese characters by statistically analyzing character frequencies and word frequencies obtained from a large Chinese dataset, includes the following steps:

[0046] S11: Traverse a large number of Chinese datasets, count the frequency of Chinese characters and sort them;

[0047] S12, use a word segmentation tool to segment the dataset, count the word frequency of Chinese characters and sort them;

[0048] S13, perform the grouping process.

[0049] S2, Calculate the number of groups, the specific steps include:

[0050] S21. Generate an identity sequence code based on the host's MAC address, IP address, hostname, username, and current time, and convert it into a binary bit sequence. The binary bit sequence is 64 bits long, of which 8 bits are check codes.

[0051] S22, calculate 64 / 2=32 groups. Since a 1-bit word occupies a separate group, a 2-bit word is divided into 31 groups.

[0052] S23, then divide the 1 bit word into 2 groups;

[0053] S3, group Chinese characters, such as Figure 1 As shown, the specific steps of the process include:

[0054] S31, Statistical analysis of character and word frequencies. Character and word frequencies are statistically analyzed using a large Chinese dataset to obtain character and word frequency tables.

[0055] S32, set the traversal loop. The loop ends when the group contains all variant characters. If all variant characters cannot be retrieved in one go, the order of the set is randomly shuffled, and the following steps are repeated:

[0056] S321, initialize the list for storing Chinese characters, dividing it into two lists, one storing 2-bit characters and the other storing 1-bit characters, and initialize... and ;in To store the list of the current groups, This is a list of Chinese characters that can form words with the current group;

[0057] S322, iterate through list 1, iterating until a character is encountered. At that time, make a judgment Is it in If so, then iterate through the next character. Otherwise, then Join Simultaneously, iterate through the first 1000 words in the word frequency list, which will be able to... Adding words to words When the frequency of the group reaches During initialization and ,renew Then proceed with the next traversal, repeating the above method until all groups have been traversed.

[0058] S323, Traverse list 2. The traversal method is the same as step S322. Since list 2 stores 1 bit word, the list is divided into two groups.

[0059] S4, the group embedding rate test module, is used to calculate the average embedding rate of groups on large datasets, such as... Figure 2 As shown, the steps include:

[0060] S41, Set parameters. Set the sliding window size and sliding distance parameters. Generally, the window size is set to 300 characters, including spaces or punctuation marks. This calculates the average embedding rate of groups in a large number of documents, and is done every 20 characters.

[0061] S42, Data Traversal. Traverse the dataset, and within each sliding window size, determine whether the embedding was successful, i.e., whether all groups were successfully embedded, and count the results.

[0062] S43, Calculate the average embedding rate. Divide the number of successful embeddings by the total number to obtain the average embedding rate.

[0063] Based on Example 1, the present invention also provides a Chinese character grouping test system based on word balance.

[0064] like Figure 3 As shown, the Chinese character grouping test system based on word balance includes the following modules:

[0065] The list traversal module is used to retrieve characters from the list of variant characters. Specifically, it reads the list of Chinese characters into memory in sequence and then passes the data to the word judgment module.

[0066] The group storage module is used to store the characters retrieved by the list traversal module. Specifically, the characters retrieved by the list traversal module are stored in the group storage module after undergoing word judgment and threshold judgment.

[0067] The vocabulary judgment module is used to determine whether the characters taken by the list traversal module can form words with the characters in the group storage module. Specifically, when the list traversal module takes a character, if it can form a word with an existing character in the group, the match fails and the next character is traversed. If it cannot, the match succeeds and the character is added to the group.

[0068] The threshold judgment module is used to determine whether the total word frequency in the group storage module reaches the threshold. Specifically, for each word retrieved, it is determined whether the total word frequency of the group exceeds (total word frequency / number of groups) + error value. If it exceeds, the word is discarded and the next word is traversed; otherwise, the word is added to the group.

[0069] like Figure 4 As shown, the Chinese character grouping test system based on word balance also includes the following modules:

[0070] The grouping judgment module is used to determine whether the input group contains all the variant characters. Specifically, it adds up the number of characters in all groups and determines whether it matches the number of variant characters contained in the variant character set. If the match is successful, a new group is generated; if the match fails, the variant character list is randomly initialized and the grouping step is completed again.

[0071] The embedding rate test module is used to calculate the average embedding rate of groups.

[0072] The embodiments of the present invention further provide application scenarios for implementing the above method embodiments and schematic processes for generating the results of the embodiments.

[0073] Given a document to be identified, the tracing tool first identifies the glyph codes of the deformed characters in the document, obtains their corresponding bit information, then identifies the group to which the character belongs, corrects errors within the group, and obtains the bit information of the group. Finally, it obtains the complete identity code according to the group order. When the number of characters in the document to be identified is small or the number of correctly identified deformed characters is small, this embodiment of the invention can provide a grouping of Chinese characters with a balanced distribution, so that the identity information can be completely extracted.

[0074] The grouping results are used as a configuration file in the tracing tool and saved as a txt file, with each line in the file representing a group. When embedding information, this grouping file needs to be read to obtain the glyph encoding of each variant character. When extracting information, the same grouping file needs to be read to obtain the corresponding bit information for each group, and finally combined to form an identity code.

[0075] Based on Embodiment 1, the present invention also provides a computer-readable storage medium including computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of the Chinese character grouping and testing method described in any of the above embodiments.

[0076] To further demonstrate the universality of this scheme, the THUCnews news text dataset was selected as the test set. A total of 1000 Chinese characters were selected as variant characters, of which 950 were 2-bit characters and the other 50 were 1-bit characters. 300 characters were selected each time, with a sliding distance of 20 characters. Equal-length grouping, which is currently widely used, was selected as the control group. The experimental results of the test set are shown in Table 1 below. It can be seen that the word-balanced grouping method improves the embedding rate by approximately 9.9% compared to equal-length grouping, and by 4.9% compared to the grouping method that only considers character frequency. Therefore, this scheme achieves good performance.

[0077] Table 1. Average embedding rates under various grouping methods.

[0078] This invention statistically analyzes a large number of Chinese datasets and calculates the word frequency. By evenly distributing the word frequency in each group, high-frequency words and low-frequency words can complement each other. The dataset is segmented into words to analyze the word frequency in the data set and avoid the occurrence of high-frequency words in the same group when grouping. (2) This invention also divides high-frequency words with simple structures into two separate groups, which carry a part of the bit information and improve the overall grouping frequency. (3) This invention also provides a method for calculating the average embedding rate. By comparing the method proposed in this invention with existing methods, a better embedding rate result is obtained.

[0079] The above description is merely a detailed explanation of preferred embodiments and principles of the present invention. For those skilled in the art, there may be changes in specific implementation methods based on the ideas provided by the present invention, and these changes should also be considered within the scope of protection of the present invention.

Claims

1. A Chinese character grouping test method based on word balance, characterized in that, Includes the following steps: S1. Obtain the Chinese text dataset, perform word segmentation, count and sort the character and word frequencies, and obtain the character frequency table and word frequency table; S2, calculate the number of groups that need to be formed; S3, group Chinese characters according to their frequency and exclude high-frequency words; the high-frequency words are those whose frequency is greater than a set threshold. S4, set the sliding window size and sliding distance, traverse the Chinese text dataset obtained in step S1, and calculate the average embedding rate of the groups; Step S1 includes the following steps: S11: Traverse the Chinese text dataset, count the frequency of Chinese characters and sort them; S12, use a word segmentation tool to segment the Chinese text dataset, count the word frequency of Chinese characters and sort them; Step S2 includes the following steps: S21. Generate an identity sequence code based on the host's MAC address, IP address, hostname, username, and current time, and convert it into a binary bit sequence with a length of length. S22, calculate k = length / 2, where k is the number of groups when only 2-bit words are used; S23, will Divide into m groups. Divided into 2 groups, so The group only needs to be divided into k-1 groups, so m=k-1; The input Chinese character encoding file contains two types of variant characters: 2-bit characters and 1-bit characters, labeled as follows: and ; Step S3 includes the following steps: S31, Calculation , The sum of the frequencies of all characters in the Chinese character set and The calculated total frequency of each group of characters is as follows: and And set the error range value. ; S32, sequential traversal of the set Chinese characters are grouped to ensure even distribution of character frequencies, and high-frequency words within the same group are eliminated by traversing the word frequency table. S33, sequential traversal of the set Chinese characters are grouped to ensure even frequency distribution, and high-frequency words within the same group are eliminated by traversing the word frequency table. The number of groups is fixed at 2. During the initialization of the character frequency table, each Chinese character is assigned an initial value. ; Step S4 includes the following steps: S41, set the window size to e and the sliding distance to g. First, extract the string with coordinates [0, e-1]. If the string contains at least one character in each group, it is successfully embedded and counted. Then, slide and extract the string with coordinates [g, e+g-1] to determine whether the embedding is successful. S42, traverse the Chinese text dataset obtained in step S1 according to step S41; S43. Divide the number of successful embeddings by the number of judgments to obtain the average embedding rate.

2. A Chinese character grouping test system based on word-word balance, used to implement the Chinese character grouping test method based on word-word balance as described in claim 1, characterized in that, The Chinese character grouping test system based on word balance includes: The list traversal module is used to retrieve characters from a list of variant characters; The grouped storage module is used to store the words retrieved by the list traversal module; The vocabulary judgment module is used to determine whether the characters taken by the list traversal module and the characters in the group storage module can form a word; The threshold judgment module is used to determine whether the sum of word frequencies in the group storage module reaches the threshold.

3. The Chinese character grouping test system based on word balance according to claim 2, characterized in that, Also includes: The grouping judgment module is used to determine whether the input group contains all the variant characters.

4. The Chinese character grouping test system based on word balance according to claim 3, characterized in that, Also includes: The embedding rate test module is used to calculate the average embedding rate of groups.

5. A computer-readable storage medium, characterized in that, It includes computer-executable instructions that, when executed by one or more processors, cause the processors to perform the steps of the Chinese character grouping test method based on word balance as described in claim 1.

Images