Character string coding and mapping method, system and device, medium and program product
By using a custom encoding polynomial and index address processing, the problem of hash collisions in embedded systems and large-scale data retrieval is solved, achieving efficient and unique string encoding and mapping, thus improving the system's stability and retrieval accuracy.
Patent Information
- Application Number
- CN202511066314.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-11-04
AI Technical Summary
Existing hash algorithms cannot meet the requirements for efficient and accurate string mapping in embedded systems and large-scale data retrieval, and they also suffer from hash collision problems.
The system uses a custom encoding polynomial to calculate the encoded value of the string and achieves a unique mapping through index address and lookup table. It leverages the properties of prime numbers to improve the uniformity of the encoded value distribution and detects and corrects conflicting encoded values or uses a chain structure to handle conflicts.
This method achieves unique string encoding within a limited encoding space, improves system stability and retrieval accuracy, reduces the probability of hash collisions, and is suitable for efficient string storage and retrieval scenarios.
Smart Images
Figure CN120893392A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer data encoding and retrieval technology, and in particular to a string encoding and mapping method, system, device, medium and program product. Background Technology
[0002] Currently, the unique identifier of a string is usually implemented using hash functions, such as the MD5 or SHA series algorithms. Although these hash algorithms can generate fixed-length hash values, they often fail to meet the requirements for efficient and accurate mapping due to their computational complexity, slow processing speed, and inability to guarantee a unique mapping for each input value, especially in embedded systems and large-scale data retrieval applications where processing efficiency and storage space requirements are high.
[0003] While hash functions offer some guarantee against collisions, in practical applications, different strings may still produce the same hash value (i.e., hash collisions). For scenarios requiring absolute uniqueness and efficient retrieval, these existing hash algorithms are insufficient. Summary of the Invention
[0004] The technical problem to be solved by this disclosure is to overcome the above-mentioned defects in the prior art and provide a string encoding and mapping method, system, device, medium and program product.
[0005] This disclosure solves the above-mentioned technical problems through the following technical solution:
[0006] This disclosure provides a string encoding and mapping method, which includes:
[0007] The encoded value of the string to be encoded is obtained by calculating the string using a custom encoding polynomial;
[0008] For each string, an index address is assigned to store the storage path corresponding to the string in the storage space of the preset size indicated by the index address, and the corresponding string is stored in the storage space indicated by the storage path.
[0009] Using the encoded value as an index, the encoded value of each string and its corresponding index address are stored in a lookup table to indicate the mapping relationship between the string, the encoded value, the index address, and the storage path.
[0010] Optionally, assigning an index address to each of the strings includes:
[0011] For the target string being encoded, check if there are other encoded values in the lookup table that are the same as the encoded value of the target string;
[0012] If it does not exist, then the index address is assigned to the target string.
[0013] Optionally, assigning an index address to each of the strings further includes:
[0014] If the lookup table contains another encoding value that is identical to the target string, then the existing string corresponding to the other encoding value is retrieved, and it is checked whether the existing string is identical to the target string. If they are not identical, the encoding value of the target string is corrected to obtain a corrected encoding value, ensuring that the corrected encoding value is unique in the lookup table. Alternatively, the target string and the existing string are linked together in a chain structure to the index address corresponding to the other encoding value, thereby achieving precise matching through a structured comparison mechanism.
[0015] Assign an index address to the target string.
[0016] Optionally, assigning an index address to each of the strings further includes:
[0017] If the existing string is the same as the target string, then the encoding process of the target string is stopped, and the encoding process of the next string is started.
[0018] Optionally, the encoding polynomial adopts the following formula:
[0019] S=s0,s1,...,s n
[0020] G(S)=(((s0*a+s1)*a+s2)*a+…+s n ) mod 2 m
[0021] Where G(S) represents the encoding polynomial, S represents the string to be encoded, and the subscripts 0-n represent the indices of the characters in the string, s0-s n The character code value represents the character code of each character in the string, where a represents a prime constant and m represents the number of bits in the encoded value.
[0022] Optionally, a can be one of 31, 131, and 65599;
[0023] m is 32;
[0024] The character codes are in UTF-8 or ASCII format.
[0025] This disclosure also provides a string encoding and mapping system, which includes: an encoding module, an index module, and a mapping module;
[0026] The encoding module is used to calculate the encoded value of the string to be encoded using a custom encoding polynomial;
[0027] The index module is used to assign an index address to each string, so as to store the storage path corresponding to the string in the storage space of the preset size indicated by the index address, and to store the corresponding string in the storage space indicated by the storage path.
[0028] The mapping module is used to store the encoded value of each string and the corresponding index address in a lookup table using the encoded value as an index, so as to indicate the mapping relationship between the string, the encoded value, the index address and the storage path.
[0029] Optionally, the indexing module is further configured to check, for the target string currently being encoded, whether there are other encoded values in the lookup table that are the same as the encoded value of the target string;
[0030] The index module is also used to allocate an index address for the target string if it does not exist.
[0031] Optionally, the indexing module is further configured to: if there are other encoded values in the lookup table that are identical to the encoded value of the target string, obtain the existing string corresponding to the other encoded value, check whether the existing string is identical to the target string, and if not, correct the encoded value of the target string to obtain a corrected encoded value, so that the corrected encoded value is unique in the lookup table; or, attach the target string and the existing string in a chain structure to the index address corresponding to the other encoded value, so as to achieve accurate matching through a structured comparison mechanism.
[0032] The index module is also used to assign an index address to the target string.
[0033] Optionally, the indexing module is further configured to stop the encoding process of the target string if the existing string is the same as the target string, so as to perform the encoding process of the next string.
[0034] Optionally, the encoding polynomial adopts the following formula:
[0035] S=s0,s1,...,s n
[0036] G(S)=(((s0*a+s1)*a+s2)*a+…+s n ) mod 2 m
[0037] Where G(S) represents the encoding polynomial, S represents the string to be encoded, and the subscripts 0-n represent the indices of the characters in the string, s0-s n The character code value represents the character code of each character in the string, where a represents a prime constant and m represents the number of bits in the encoded value.
[0038] Optionally, a can be one of 31, 131, and 65599;
[0039] m is 32;
[0040] The character codes are in UTF-8 or ASCII format.
[0041] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein the processor executes the computer program to implement the aforementioned string encoding and mapping method.
[0042] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned string encoding and mapping method.
[0043] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned string encoding and mapping method.
[0044] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.
[0045] The positive improvements of this disclosure are as follows: A pre-defined fixed-length encoding value is obtained by encoding the string using a custom encoding polynomial. A prime number with good hashing properties is selected as the polynomial base, improving the uniformity of the encoding value distribution and thus reducing the probability of hash collisions. During encoding, it checks whether the encoding value has already been used. If it is detected that the encoding value has been used for other strings, the encoding value is corrected to make it unique. Alternatively, without changing the encoding value, multiple conflicting strings are linked to the index address of the encoding value in a chain structure, and a structured comparison mechanism is used to achieve accurate matching. This ensures the unique encoding of strings under limited encoding space, improving system stability and retrieval accuracy. Attached Figure Description
[0046] Figure 1 A flowchart of a string encoding and mapping method provided in Embodiment 1 of this disclosure;
[0047] Figure 2A flowchart illustrating a specific implementation of step S12 of a string encoding and mapping method provided in Embodiment 1 of this disclosure;
[0048] Figure 3 A flowchart illustrating another specific implementation of step S12 of the string encoding and mapping method provided in Embodiment 1 of this disclosure;
[0049] Figure 4 This is a schematic diagram of a string encoding and mapping system provided in Embodiment 2 of this disclosure;
[0050] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this disclosure. Detailed Implementation
[0051] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.
[0052] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not be construed as an unnecessary limitation. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.
[0053] Example 1
[0054] Figure 1 A flowchart illustrating a string encoding and mapping method provided as an exemplary embodiment of this disclosure is provided. The string encoding and mapping method includes:
[0055] S11. Calculate the encoding value of the string to be encoded using a custom encoding polynomial.
[0056] S12. For each string, allocate an index address to store the storage path corresponding to the string in the storage space of the preset size indicated by the index address, and store the corresponding string in the storage space indicated by the storage path.
[0057] S13. Using the encoded value as an index, store the encoded value of each string and its corresponding index address in a lookup table to indicate the mapping relationship between the string, the encoded value, the index address, and the storage path.
[0058] For strings of arbitrary length, a pre-defined encoding polynomial is used to obtain an encoded value with a fixed number of bits, preferably 32 bits (4 bytes). By selecting a prime number with good hashing properties as the base (constant) of the polynomial, the uniformity of the encoded value distribution is improved, thereby reducing the probability of hash collisions. Through a carefully designed encoding polynomial, it is ensured that each string receives a unique encoding within a limited encoding space.
[0059] In addition to storing the string itself (the original string), the storage space indicated by the storage path can also store related content corresponding to the string, such as records in the database, the storage path of the file to which it belongs, or other metadata.
[0060] Each encoded value stored in the lookup table is unique, and its index address is also unique. This unique encoded value allows for quick retrieval of the corresponding unique address information, further enabling the search of string-related content. Specifically, the lookup table can be used to find the corresponding index address, then the storage path of the string can be obtained from the index address, and finally, the string itself and its related content can be retrieved from the storage path. The lookup table allows for quick checking for conflicts with existing string codes and efficient retrieval of the string-related content corresponding to the encoded value. It achieves efficient string retrieval and content mapping while ensuring the uniqueness of the encoding and the accuracy of the query. This method is suitable for scenarios requiring efficient string storage and retrieval, especially in applications with large data volumes and high demands for storage and retrieval speed. It is applicable to various embedded and resource-constrained environments.
[0061] In this embodiment, a pre-defined fixed-length encoding value is obtained by encoding the string using a custom encoding polynomial. A prime number with good hashing properties is selected as the base of the polynomial to improve the uniformity of the encoding value distribution, thereby reducing the probability of hash collisions. During encoding, it is detected whether the encoding value has been occupied. If it is detected that the encoding value has been used for other strings, the encoding value is corrected to make it unique. Alternatively, if the encoding value remains unchanged, multiple conflicting strings are linked to the index address of the encoding value in a chain structure, and accurate matching is achieved through a structured comparison mechanism. This ensures the unique encoding of strings under the condition of limited encoding space, improving system stability and retrieval accuracy.
[0062] In one embodiment, refer to Figure 2 Step S12, "assigning an index address to each string," includes:
[0063] S121. For the target string of the current encoding process, check if there are other encoding values in the lookup table that are the same as the encoding value of the target string.
[0064] S122. If it does not exist, then allocate an index address for the target string.
[0065] In one embodiment, refer to Figure 3 Step S12, "assigning an index address to each string," also includes:
[0066] S123. If there are other encoded values in the lookup table that are the same as the encoded value of the target string, then obtain the existing string corresponding to the other encoded value, check whether the existing string is the same as the target string. If they are not the same, then modify the encoded value of the target string to obtain the modified encoded value so that the modified encoded value is unique in the lookup table. Alternatively, the target string and the existing string are linked to the index address corresponding to the other encoded value in a chain structure to complete the accurate matching through the structured comparison mechanism.
[0067] S124. Allocate an index address for the target string.
[0068] While the custom-defined encoding polynomial exhibits good distribution, it suffers from limitations in encoding space (32 bits, maximum 2^32). 32 -1), theoretically, there is still a possibility that different strings could have the same encoding result (similar to a hash collision). To ensure the uniqueness of the encoded value, if a collision occurs (i.e., two different strings map to the same encoded value), the encoded value can be corrected (e.g., using different parameter values as the polynomial base or introducing a second-level hash function to re-encode the conflicting strings), chaining (keeping the encoded value unchanged), or other collision solutions can be used to handle the situation.
[0069] The chaining method is adopted, which means that, without changing the encoded value, multiple conflicting strings are linked to the index address of the encoded value in a chain structure, and accurate matching is achieved through a structured comparison mechanism.
[0070] This embodiment can be extended to a wider range of coding systems to adapt to application needs of different scales.
[0071] In one embodiment, step S12, "assigning an index address to each string," further includes:
[0072] If a string already exists that is identical to the target string, the encoding process for the target string is stopped, and the encoding process for the next string begins.
[0073] In one embodiment, the encoded polynomial is expressed using the following formula:
[0074] S=s0,s1,...,s n
[0075] G(S)=(((s0*a+s1)*a+s2)*a+…+sn ) mod 2 m
[0076] Where G(S) represents the encoding polynomial, S represents the string to be encoded, and the subscripts 0-n represent the indices of the characters in the string, s0-s... n This represents the code value of each character in the string, where 'a' represents a prime constant and 'm' represents the number of bits in the encoded value.
[0077] In the event of an encoding conflict, different parameter values (prime constant a) can be used as the polynomial base or a two-level hash function can be introduced to re-encode the conflicting string in order to correct the encoding value.
[0078] It can record the number and type of encoding collisions. If collisions are frequent, the collision probability can be adaptively adjusted by automatically selecting different parameter values of the prime constant a (dynamically adjusted) or using a backup hash function.
[0079] In one embodiment, a is one of 31, 131, and 65599.
[0080] m is 32.
[0081] The character code uses UTF-8 or ASCII.
[0082] Preferably, the character code adopts UTF-8 code.
[0083] Example 2
[0084] Corresponding to the aforementioned string encoding and mapping method embodiments, this disclosure also provides embodiments of string encoding and mapping systems.
[0085] Figure 4 This is a schematic diagram of a string encoding and mapping system provided for an exemplary embodiment of the present disclosure. The system includes: an encoding module 1, an indexing module 2, and a mapping module 3.
[0086] Encoding module 1 is used to calculate the encoding value of the string to be encoded using a custom encoding polynomial.
[0087] Index module 2 is used to assign an index address to each string, so as to store the storage path corresponding to the string in the storage space of the preset specification indicated by the index address, and to store the corresponding string in the storage space indicated by the storage path.
[0088] The mapping module 3 is used to store the encoding value of each string and its corresponding index address in a lookup table, using the encoded value as an index, to indicate the mapping relationship between the string, the encoded value, the index address, and the storage path.
[0089] In this embodiment, a pre-defined fixed-length encoding value is obtained by encoding the string using a custom encoding polynomial. A prime number with good hashing properties is selected as the base of the polynomial to improve the uniformity of the encoding value distribution, thereby reducing the probability of hash collisions. During encoding, it is detected whether the encoding value has been occupied. If it is detected that the encoding value has been used for other strings, the encoding value is corrected to make it unique. Alternatively, if the encoding value remains unchanged, multiple conflicting strings are linked to the index address of the encoding value in a chain structure, and accurate matching is achieved through a structured comparison mechanism. This ensures the unique encoding of strings under the condition of limited encoding space, improving system stability and retrieval accuracy.
[0090] In one embodiment, the index module 2 is also used to check whether there are other encoded values in the lookup table that are the same as the encoded value of the target string for the current encoding processing.
[0091] Index module 2 is also used to allocate an index address for the target string if it does not exist.
[0092] In one embodiment, the index module 2 is further configured to, if there are other encoded values in the lookup table that are the same as the encoded value of the target string, obtain the existing string corresponding to the other encoded value, check whether the existing string is the same as the target string, and if they are not the same, modify the encoded value of the target string to obtain the modified encoded value so that the modified encoded value is unique in the lookup table. Then, the target string and the existing string are linked in a chain structure to the index address corresponding to the other encoded value so as to complete the accurate matching through the structured comparison mechanism.
[0093] Index module 2 is also used to assign an index address to the target string.
[0094] In one embodiment, the index module 2 is further configured to stop the encoding process of the target string if an existing string is identical to the target string, so as to proceed with the encoding process of the next string.
[0095] In one embodiment, the encoded polynomial is expressed using the following formula:
[0096] S=s0,s1,...,s n
[0097] G(S)=(((s0*a+s1)*a+s2)*a+…+s n ) mod 2 m
[0098] Where G(S) represents the encoding polynomial, S represents the string to be encoded, and the subscripts 0-n represent the indices of the characters in the string, s0-s... nThis represents the code value of each character in the string, where 'a' represents a prime constant and 'm' represents the number of bits in the encoded value.
[0099] In one embodiment, a is one of 31, 131, and 65599.
[0100] m is 32.
[0101] The character code uses UTF-8 or ASCII.
[0102] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.
[0103] Example 3
[0104] Figure 5 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the string encoding and mapping method described in any of the above embodiments. Figure 5 The electronic device 90 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.
[0105] like Figure 5 As shown, the electronic device 90 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including memory 92 and processor 91).
[0106] Bus 93 includes a data bus, an address bus, and a control bus.
[0107] The memory 92 may include volatile memory, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.
[0108] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) program module 924, such program module 924 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.
[0109] The processor 91 executes various functional applications and data processing by running computer programs stored in the memory 92, such as the string encoding and mapping methods provided in any of the above embodiments.
[0110] Electronic device 90 can also communicate with one or more external devices 94 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 95. Furthermore, electronic device 90 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 96. As shown, network adapter 96 communicates with other modules of electronic device 90 via bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.
[0111] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.
[0112] Example 4
[0113] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the string encoding and mapping method provided in any of the above embodiments.
[0114] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.
[0115] Example 5
[0116] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the string encoding and mapping method described in any of the preceding embodiments.
[0117] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.
[0118] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.
Claims
1. A string encoding and mapping method, characterized in that, The string encoding and mapping method includes: The encoded value of the string to be encoded is obtained by calculating the string using a custom encoding polynomial; For each string, an index address is assigned to store the storage path corresponding to the string in the storage space of the preset size indicated by the index address, and the corresponding string is stored in the storage space indicated by the storage path. Using the encoded value as an index, the encoded value of each string and its corresponding index address are stored in a lookup table to indicate the mapping relationship between the string, the encoded value, the index address, and the storage path.
2. The string encoding and mapping method as described in claim 1, characterized in that, Assigning an index address to each of the strings includes: For the target string being encoded, check if there are other encoded values in the lookup table that are the same as the encoded value of the target string; If it does not exist, then the index address is assigned to the target string.
3. The string encoding and mapping method as described in claim 2, characterized in that, Assigning an index address to each of the strings also includes: If there are other encoded values in the lookup table that are the same as the encoded value of the target string, then the existing string corresponding to the other encoded value is obtained, and it is checked whether the existing string is the same as the target string. If they are not the same, the encoded value of the target string is corrected to obtain a corrected encoded value, so that the corrected encoded value is unique in the lookup table. Alternatively, the target string and the existing string are linked to the index address corresponding to the other encoded value in a chain structure to complete accurate matching through a structured comparison mechanism. Assign an index address to the target string.
4. The string encoding and mapping method as described in claim 3, characterized in that, Assigning an index address to each of the strings also includes: If the existing string is the same as the target string, then the encoding process of the target string is stopped, and the encoding process of the next string is started.
5. The string encoding and mapping method as described in claim 1, characterized in that, The encoding polynomial adopts the following formula: S=s0,s1,...,s n G(S)=(((s0*a+s1)*a+s2)*a+…+s n ) mod 2 m Where G(S) represents the encoding polynomial, S represents the string to be encoded, and the subscripts 0-n represent the indices of the characters in the string, s0-s n The character code value represents the character code of each character in the string, where a represents a prime constant and m represents the number of bits in the encoded value.
6. The string encoding and mapping method as described in claim 5, characterized in that, 'a' is one of 31, 131, and 65599; m is 32; The character codes are in UTF-8 or ASCII format.
7. A string encoding and mapping system, characterized in that, The string encoding and mapping system includes: an encoding module, an indexing module, and a mapping module; The encoding module is used to calculate the encoded value of the string to be encoded using a custom encoding polynomial; The index module is used to assign an index address to each string, so as to store the storage path corresponding to the string in the storage space of the preset size indicated by the index address, and to store the corresponding string in the storage space indicated by the storage path. The mapping module is used to store the encoded value of each string and the corresponding index address in a lookup table using the encoded value as an index, so as to indicate the mapping relationship between the string, the encoded value, the index address and the storage path.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the string encoding and mapping method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the string encoding and mapping method as described in any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the string encoding and mapping method as described in any one of claims 1-6.