Control character filtering method and device based on font ID and conforming to Mongolian encoding standard, readable storage medium and equipment

Abstract

The invention provides a font ID-based control character filtering method and device conforming to a Mongolian encoding standard, a readable storage medium and equipment. The method comprises the following steps: S1, inputting a Mongolian word; s2, counting the number of Mongolian word control characters stored in the MWord; s3, setting a counter N to be 1; s4, judging whether the count of N exceeds the number of the control characters or not, if not, executing a loop body, and obtaining font ID strings corresponding to Mongolian words before and after the Nth control character is deleted; and S5, whether the font ID strings corresponding to the Mongolian words before and after deleting the Nth control character are equal or not is judged, if yes, the Nth control character of the Mongolian words stored in the MWord is deleted, and if not, the Nth control character of the Mongolian words stored in the MWord is not deleted. According to the method, redundant control characters in the Mongolian text resources can be filtered out, system resources are saved, the simplicity of the Mongolian text resources is kept, and meanwhile the accuracy of Mongolian characters is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer word processing, in particular, to a control symbol filtering method and device based on glyph ID and in line with the Mongolian coding standard, a readable storage medium and equipment. BACKGROUND

[0002] The international standard for Mongolian coding was approved by ISO / IEC JTC 1 / SC2 in February 2000, ending the era of no unified coding for Mongolian. In 2010, the standard "GB / T 25914-2010 Information Technology Traditional Mongolian Nominal Characters, Deformation Display Characters and Control Characters Usage Rules" was established, which is the first Mongolian conversion rule standard.

[0003] Since 2010, the development and production of Mongolian OpenType font must comply with the Unicode standard and national standard. In particular, GB / T 25914-2010 and GB / T 25914-2023, which is revised on the basis of the former, were revised in 2016. After the seven major Mongolian font development companies upgraded their own fonts based on this revised standard, the rules of the major Mongolian OpenType fonts used on the market have basically been unified. GB / T 25914-2023, the revised standard of GB / T 25914-2010, will be implemented from June 1, 2024, and GB / T 25914-2010 will be abolished. Future Mongolian OpenType fonts must comply with GB / T 25914-2023.

[0004] During the input process of Mongolian text, the developers of input methods and Mongolian input personnel do not consider the "contextual relationship" between Mongolian characters to input the correct Mongolian glyph, but use control symbols to forcibly output the correct Mongolian glyph. Although this can ensure the correctness of the Mongolian glyph, there are many "redundant" control symbols in Mongolian words that should not be used. As shown in Table 1, Mongolian coding string 1 and 2 display the same Mongolian glyph after being processed by the text engine. This shows that the control symbol 180C in Mongolian coding string 1 is redundant. Table 2 shows that Mongolian coding string 1 and 2 display different Mongolian glyphs for character 182D after being processed by the text engine. This shows that the control symbol 180B in Mongolian coding string 1 is necessary.

[0005] Table 1 Control symbols that can be filtered Table 2 Control symbols that cannot be filtered These redundant control characters can severely impact the usability of Mongolian text resources. Filtering these redundant control characters is a crucial task in Mongolian text preprocessing to ensure their quality. Therefore, filtering control characters from Mongolian resources is a pressing issue in Mongolian information processing.

[0006] Currently, there's no method for filtering control characters. The reason: Before the release of GB / T 25914-2023, Mongolian font developers needed to add many "custom conversion rules" to their fonts, even if they complied with GB / T 25914-2010. GB / T 25914-2010 also includes many conversion rules not covered by the standard, meaning there's no unified standard for these rules. Each font manufacturer added their own interpretation of these rules (their own "conversion rules") to their fonts. This led to inconsistencies in conversion rules between fonts from different font manufacturers.

[0007] The release of GB / T 25914-2023 fully covers Mongolian conversion rules. Therefore, regardless of the Mongolian character library manufacturer, it can filter out "redundant" control characters in Mongolian resources. Summary of the Invention

[0008] In view of this, the present application provides a method and apparatus for filtering control characters that comply with Mongolian encoding standards based on glyph IDs, so as to filter out redundant control characters in Mongolian encoding strings.

[0009] To achieve the above objectives, the technical solutions adopted in this application are as follows: On the one hand, it provides a control character filtering method based on glyph ID that complies with Mongolian encoding standards, including: S1: Input a Mongolian word and save the input Mongolian word in the variable MWord; the encoding of the Mongolian word complies with the GB / T 25914-2023 standard; S2: Counting the number of Mongolian word control characters stored in the MWord, and storing the statistical result in a variable M; S3: Set counter N=1, counter N is the loop variable; S4: Determine whether the count of N exceeds the number of control characters. If so, output the Mongolian word stored in the variable MWord and terminate the process. Otherwise, execute the loop body, including: storing the word formed by pre-deleting the Nth control character from the Mongolian word in MWord in the intermediate result variable MidWord, and then obtaining the glyph ID string corresponding to the Mongolian word stored in MidWord through the text rendering engine; at the same time, obtaining the glyph ID string corresponding to the Mongolian word stored in MWord through the text rendering engine; S5: comparing the glyph ID string corresponding to the Mongolian word stored in MidWord with the glyph ID string corresponding to the Mongolian word stored in MWord to see if they are equal; if they are equal, it indicates that the Nth control character is a redundant control character, and the Nth control character storing the Mongolian word in MWord is deleted; if they are not equal, it indicates that the Nth control character is necessary, and the Nth control character storing the Mongolian word in MWord is not deleted; S6: Let N=N+1; S7: Repeat steps S4-S6 until N>M, all control characters are processed, and the loop ends; S8: Finally, the Mongolian word stored in the variable MWord is output after the redundant control characters are filtered out.

[0010] Furthermore, the text rendering engine includes Harfbuzz and Directwrite.

[0011] In addition, the present application provides a control character filtering device based on glyph ID and compliant with Mongolian encoding standards, including: An input unit, configured to input a Mongolian word and store the input Mongolian word in a variable MWord; the encoding of the Mongolian word complies with the GB / T 25914-2023 standard; a counting unit, configured to count the number of Mongolian word control characters stored in the MWord and store the counting result in a variable M; The counting unit is used to set the counter N=1, where the counter N is a loop variable; A judgment execution unit is used to determine whether the count of N exceeds the number of control characters. If so, the Mongolian word stored in the variable MWord is output and the process ends; otherwise, a loop body is executed, including: storing the word formed after deleting the Nth control character from the Mongolian word in MWord in the intermediate result variable MidWord, then obtaining the glyph ID string corresponding to the Mongolian word stored in MidWord through the text rendering engine, and simultaneously obtaining the glyph ID string corresponding to the Mongolian word stored in MWord through the text rendering engine; a comparison and deletion unit, configured to compare a glyph ID string corresponding to a Mongolian word stored in MidWord with a glyph ID string corresponding to a Mongolian word stored in MWord to determine whether they are equal; if they are equal, indicating that the Nth control character is a redundant control character, the Nth control character storing the Mongolian word in MWord is deleted; if they are not equal, indicating that the Nth control character is necessary, the Nth control character storing the Mongolian word in MWord is not deleted; Addend unit, used to make N=N+1; The return unit is used to repeatedly execute steps S4-S6 until N>M, all control characters are processed, and the loop ends; The output unit is used to finally output the Mongolian word stored in the variable MWord after filtering out redundant control characters.

[0012] Furthermore, the text rendering engine includes Harfbuzz and Directwrite.

[0013] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the control character filtering method based on glyph ID and compliant with the Mongolian encoding standard of the present application are implemented.

[0014] According to another aspect of the present application, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for filtering control characters based on glyph IDs and conforming to Mongolian encoding standards of the present application are implemented.

[0015] Compared with the prior art, the present invention has the following advantages: 0. The control character filtering method of the present application can filter out "redundant" control characters in Mongolian text resources, thereby saving system resources and keeping Mongolian text resources concise while ensuring the accuracy of Mongolian character shapes.

[0016] 1. This application method is simple and easy, without complicated operation procedures. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 This is a flow chart of a control character filtering method based on glyph ID that complies with Mongolian encoding standards.

[0019] Figure 2 This is a structural block diagram of a control character filtering device based on glyph ID and compliant with Mongolian encoding standards.

[0020] Figure 3 This is an example diagram of glyph ID in the Mongolian character library.

[0021] Figure 4This is a screenshot of the code for counting the number of control characters in Mongolian words in the specific implementation method of this application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0023] like Figure 1 As shown, the control character filtering method based on glyph ID and compliant with Mongolian encoding standards includes: S1: Input a Mongolian word and save the input Mongolian word in the variable MWord; the encoding of the Mongolian word complies with the GB / T 25914-2023 standard; Mongolian conversion rules are based on "words," so Mongolian control characters are also filtered on a word-by-word basis. The input Mongolian word is saved in the variable MWord for later use in the filtering process.

[0024] In this embodiment, it is assumed that the encoding string of the input Mongolian word is: / U+182A / U+1820 / U+1832 / U+1824 / U+180C / U+1830 / U+1826 / U+180B / U+182C / U+1821.

[0025] S2: Count the number of Mongolian word control characters in the MWord and save the statistical result in a variable M; By comparing the codes of each character in a loop, the number of character codes equal to U+180B, U+180C, U+180D, and U+180F is counted, and the statistical results are saved in the variable M.

[0026] For the specific code implementation process of counting the number of control characters, please refer to Figure 4 .

[0027] S3: Set counter N=1, counter N is the loop variable; The counter is incremented by 1 at the end of each loop.

[0028] S4: Determine whether the count of N exceeds the number of control characters. If so, output the Mongolian word stored in the variable MWord and terminate the process. Otherwise, execute the loop body, including: storing the word formed by pre-deleting the Nth control character from the Mongolian word in MWord in the intermediate result variable MidWord, and then obtaining the glyph ID string corresponding to the Mongolian word stored in MidWord through the text rendering engine; at the same time, obtaining the glyph ID string corresponding to the Mongolian word stored in MWord through the text rendering engine; Specifically, the text rendering engines include Harfbuzz and Directwrite.

[0029] Each glyph in the Mongolian character library has its own unique ID number. The reason why the encoding of Mongolian characters can finally be displayed as a Mongolian glyph is because of the IDs of these glyphs used in the intermediate process. When each Mongolian word is displayed on the screen, the 'Mongolian encoding string' needs to pass through the text rendering engine of the operating system, and the conversion rules of 'Mongolian encoding to Mongolian glyphs' are interpreted in the text rendering engine, and then the 'Mongolian glyph ID string' is obtained, and finally the glyph graphics corresponding to the glyph ID are drawn to obtain the display result. Figure 3 The Mongolian font glyph ID is displayed. The “#xxx” above each glyph is the glyph ID.

[0030] Assume that the encoding string of the above Mongolian word is in a certain font library. During the first loop, after deleting the first control character U+180C, the glyph IDs obtained by the rendering engine are: 369, 674, 300, 663, 317, 513.

[0031] Assuming that the encoding string of the above Mongolian word is in a certain font library, in the second loop, after deleting the second control character U+180B, the glyph IDs obtained by the rendering engine are: 369, 674, 300, 663, 318, 513.

[0032] Assuming that the encoding string of the above Mongolian word is in a certain font library, without deleting the control characters, the glyph IDs obtained by the rendering engine are: 369, 674, 300, 663, 317, 513.

[0033] S5: comparing the glyph ID string corresponding to the Mongolian word stored in MidWord with the glyph ID string corresponding to the Mongolian word stored in MWord to see if they are equal; if they are equal, it indicates that the Nth control character is a redundant control character, and the Nth control character storing the Mongolian word in MWord is deleted; if they are not equal, it indicates that the Nth control character is necessary, and the Nth control character storing the Mongolian word in MWord is not deleted; In the first iteration of the Mongolian word encoding string in a certain font library, the glyph IDs before and after the first control character U+180C is deleted (369, 674, 300, 663, 317, 513) are exactly the same. Therefore, the first control character U+180C can be deleted because it is redundant.

[0034] The encoding string of the above Mongolian word is in a certain font library. During the second loop, the glyph IDs before and after the second control character U+180B are deleted (369, 674, 300, 663, 317, 513) and (369, 674, 300, 663, 318, 513). 317 and 318 are different, so the second control character U+180B cannot be deleted and is a valid control character.

[0035] S6: Let N=N+1; S7: Repeat steps S4-S6 until N>M, all control characters are processed, and the loop ends; S8: Finally, the Mongolian word stored in the variable MWord is output after the redundant control characters are filtered out.

[0036] The encoding string for the Mongolian word input is: / U+182A / U+1820 / U+1832 / U+1824 / U+180C / U+1830 / U+1826 / U+180B / U+182C / U+1821. After two cycles, the final filtered result is: / U+182A / U+1820 / U+1832 / U+1824 / U+1830 / U+1826 / U+180B / U+182C / U+1821.

[0037] This application obtains two "Mongolian glyph ID strings" before and after deleting a control character. By comparing the ID values ​​in these two "Mongolian glyph ID strings," it determines whether the control character is "redundant." As shown in Table 3, the glyph ID string after filtering out the control character "180B" in the first encoding string is equal to the glyph ID string of the second encoding string without 180B. Therefore, the control character "180B" is redundant and can be filtered. Note: The glyph of the control character is not displayed on the screen, that is, there is no graphic for "124" in the glyph string.

[0038] Table 3 Control character filtering example As shown in Table 4, the glyph ID string after filtering out the control character "180B" in the first code string is not equal to the glyph ID string in the second code string without 180B. That is, after filtering out the control character "180B," the glyph ID of the character "182D" changes from "311" to "312." Therefore, the control character "180B" cannot be filtered. Note: The glyph of the control character is not displayed on the screen, that is, there is no graphic for "124" in the glyph string.

[0039] Table 4 Control character filtering example This application can be presented as a DLL (Dynamic Link Library). To use this application, you can directly call the interface using a high-level computer language that supports DLLs (C++, C#, etc.). The interface sends the Mongolian text that requires filter control characters as a parameter, and returns Mongolian text that contains filter control characters that comply with the GB / T25914-2023 standard.

[0040] like Figure 2 As shown, the control character filtering device based on the glyph ID and compliant with the Mongolian encoding standard includes: An input unit 210 is used to input a Mongolian word and save the input Mongolian word in a variable MWord; the encoding of the Mongolian word complies with the GB / T 25914-2023 standard; A counting unit 220 counts the number of Mongolian word control characters in the MWord and stores the counted result in a variable M; The counting unit 230 is used to set the counter N=1, where the counter N is a loop variable; The judgment execution unit 240 is used to determine whether the count N exceeds the number of control characters. If so, the Mongolian word stored in the variable MWord is output and the process ends. Otherwise, a loop body is executed, including: storing the word formed by pre-deleting the Nth control character from the Mongolian word in MWord in the intermediate result variable MidWord, then obtaining the glyph ID string corresponding to the Mongolian word stored in MidWord through the text rendering engine, and simultaneously obtaining the glyph ID string corresponding to the Mongolian word stored in MWord through the text rendering engine; a comparison and deletion unit 250 for comparing the glyph ID string corresponding to the Mongolian word stored in MidWord with the glyph ID string corresponding to the Mongolian word stored in MWord to determine whether they are equal; if they are equal, it indicates that the Nth control symbol is a redundant control symbol, and the Nth control symbol storing the Mongolian word in MWord is deleted; if they are not equal, it indicates that the Nth control symbol is necessary, and the Nth control symbol storing the Mongolian word in MWord is not deleted; Addend unit 260, used to set N=N+1; Return unit 270 is used to repeatedly execute steps S4-S6 until N>M, all control characters are processed, and the loop ends; The output unit 280 is used to finally output the Mongolian word stored in the variable MWord after the redundant control characters are filtered out.

[0041] Furthermore, the text rendering engine includes Harfbuzz and Directwrite.

[0042] As for the device embodiment, since it basically corresponds to the method embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the method embodiment. The device embodiment described above is only illustrative. The units described as separate components may or may not be physically separated.

[0043] The components may or may not be physical units, that is, they may be located in one place or distributed across multiple network units, such as a server and a client. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present embodiment without inventive effort.

[0044] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the control character filtering method based on glyph ID and compliant with the Mongolian encoding standard of the present application are implemented.

[0045] According to another aspect of the present application, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for filtering control characters based on glyph IDs and conforming to Mongolian encoding standards of the present application are implemented.

[0046] The control character filtering method of the present application can filter out "redundant" control characters in Mongolian text resources, and can also ensure the accuracy of Mongolian character shapes and the cleanliness of text resources.

[0047] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A control character filtering method that complies with Mongolian coding standards based on glyph ID, characterized in that: include: S1: Input a Mongolian word and save the input Mongolian word in the variable MWord; The encoding of the Mongolian words complies with the GB / T 25914-2023 standard; S2: Counting the number of Mongolian word control characters stored in the MWord, and storing the statistical result in a variable M; S3: Set counter N=1, counter N is the loop variable; S4: Determine whether the count of N exceeds the number of control characters. If so, output the Mongolian word stored in the variable MWord and end the process. Otherwise, a loop body is executed, including: storing the word formed by pre-deleting the Nth control character from the Mongolian word in MWord in an intermediate result variable MidWord, and then obtaining the glyph ID string corresponding to the Mongolian word stored in MidWord through a text rendering engine; at the same time, obtaining the glyph ID string corresponding to the Mongolian word stored in MWord through the text rendering engine; S5: comparing the glyph ID string corresponding to the Mongolian word stored in MidWord with the glyph ID string corresponding to the Mongolian word stored in MWord to see if they are equal; if they are equal, it indicates that the Nth control character is a redundant control character, and the Nth control character storing the Mongolian word in MWord is deleted; if they are not equal, it indicates that the Nth control character is necessary, and the Nth control character storing the Mongolian word in MWord is not deleted; S6: Let N=N+1; S7: Repeat steps S4-S6 until N>M, all control characters are processed, and the loop ends; S8: Finally, the Mongolian word stored in the variable MWord is output after the redundant control characters are filtered out.

2. The control character filtering method according to the Mongolian coding standard based on the glyph ID as claimed in claim 1, wherein: The text rendering engines include Harfbuzz and Directwrite.

3. A control character filtering device based on glyph ID that complies with Mongolian coding standards, characterized in that: include: The input unit is used to input a Mongolian word and save the input Mongolian word in the variable MWord; The encoding of the Mongolian words complies with the GB / T 25914-2023 standard; a counting unit, configured to count the number of Mongolian word control characters stored in the MWord and store the counted result in a variable M; The counting unit is used to set the counter N=1, where the counter N is a loop variable; A judgment execution unit is used to judge whether the count of N exceeds the number of control characters. If so, the Mongolian word stored in the variable MWord is output and the process ends; Otherwise, a loop body is executed, including: storing the word formed by pre-deleting the Nth control character from the Mongolian word in MWord in an intermediate result variable MidWord, then obtaining the glyph ID string corresponding to the Mongolian word stored in MidWord through a text rendering engine, and simultaneously obtaining the glyph ID string corresponding to the Mongolian word stored in MWord through a text rendering engine; a comparison and deletion unit, configured to compare a glyph ID string corresponding to a Mongolian word stored in MidWord with a glyph ID string corresponding to a Mongolian word stored in MWord to determine whether they are equal; if they are equal, indicating that the Nth control character is a redundant control character, the Nth control character storing the Mongolian word in MWord is deleted; if they are not equal, indicating that the Nth control character is necessary, the Nth control character storing the Mongolian word in MWord is not deleted; Addend unit, used to make N=N+1; The return unit is used to repeatedly execute steps S4-S6 until N>M, all control characters are processed, and the loop ends; The output unit is used to finally output the Mongolian word stored in the variable MWord after filtering out redundant control characters.

4. The control symbol filtering device according to the Mongolian coding standard based on glyph ID as claimed in claim 3, characterized in that: The text rendering engines include Harfbuzz and Directwrite.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the control character filtering method based on glyph ID and conforming to the Mongolian encoding standard are implemented as claimed in claim 1 or 2.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method for filtering control characters based on glyph ID and conforming to the Mongolian encoding standard as claimed in claim 1 or 2 are implemented.

Images