A protocol name resolution method and device based on hash mapping
By pre-registering a set of protocol names and utilizing a hash mapping structure, the problem of creating temporary objects for protocol name resolution in high-concurrency network communication is solved, improving processing speed and system stability, and achieving efficient protocol name matching and resource management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING EBAY INTERACTIVE NETWORK TECHNOLOGY CO LTD
- Filing Date
- 2026-04-17
- Publication Date
- 2026-06-26
Smart Images
Figure CN122293771A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of network communication technology, and in particular to a method and apparatus for resolving protocol names based on hash mapping. Background Technology
[0002] In network communication systems, protocol name resolution is a crucial step in message processing. When the network layer receives a protocol message, it needs to extract the protocol name field from the message header and convert it into a recognizable string identifier for subsequent protocol routing, message distribution, and service processing.
[0003] Existing technologies primarily employ a direct conversion scheme from byte arrays to strings. Specifically, during each protocol message decoding, the byte array containing the protocol name is converted into a string object by calling the string constructor method in the Java standard library. This string object is then used for protocol routing determination, typically by comparing the protocol name content using the `equals()` method to identify the corresponding business handler. Some optimization schemes utilize string constant pooling, using the `String.intern()` method to include dynamically created strings in the JVM constant pool, enabling the reuse of string instances. Other high-performance network frameworks employ predefined protocol enumeration types, mapping the byte array content to the corresponding enumeration instance during parsing, and then obtaining the enumeration name as the protocol identifier.
[0004] While the existing technical solutions described above can meet basic protocol parsing requirements at the functional level, their performance and resource utilization efficiency suffer from significant bottlenecks in high-concurrency, high-throughput network communication scenarios, making it difficult to support business pressures of millions of QPS. Frequent object creation leads to immense garbage collection pressure, severely impacting system throughput. In high-concurrency network communication scenarios, each protocol message parsing requires converting a byte array into a string object using `new String(protoNameBytes)`. This method frequently creates temporary objects at runtime, generating millions of short-lived objects per second in scenarios with millions of QPS. These objects quickly fill the young generation memory, frequently triggering Young GC. During GC pauses, application threads are interrupted, causing a precipitous drop in system throughput. In extreme cases, it can even trigger a full heap GC, causing second-level service interruptions and severely impacting business continuity. Therefore, how to eliminate temporary object creation during protocol name parsing has become an urgent problem to solve. Summary of the Invention
[0005] This application provides a protocol name resolution method and apparatus based on hash mapping, which avoids the creation of temporary objects during protocol name resolution, reduces frequent memory allocation and release operations, and lowers runtime resource consumption. This application pre-registers a set of protocol names and stores the byte arrays of each protocol name in a resource space pool. It then calculates the corresponding hash value using a hash algorithm and constructs a mapping structure between the hash value and the protocol name string. Simultaneously, it caches the protocol name string. When obtaining a protocol message and resolving the protocol name, it can directly query the mapping structure based on the hash value to complete the protocol name matching, eliminating the need to repeatedly generate protocol name-related data objects and improving the processing speed of protocol name resolution. The use of the hash algorithm ensures the accuracy and reliability of the protocol name matching results, avoiding data processing errors caused by the creation of temporary objects. The cached protocol name string can be reused multiple times, reducing repetitive data calculation and processing steps and optimizing runtime efficiency. The centralized storage of the protocol name byte array in the resource space pool standardizes the management of protocol name-related data, avoids system resource consumption caused by scattered data storage, makes the entire protocol name resolution process more streamlined and smooth, and improves the stability of long-term operation.
[0006] In a first aspect, embodiments of this application provide a protocol name resolution method based on hash mapping, the method comprising: In response to the system startup command, obtain the set of pre-registered protocol names; The byte arrays of each protocol name are placed into the resource space pool, and the hash values are calculated based on the hash algorithm. Establish a mapping structure from hash values to protocol name strings, and cache the protocol name strings; Obtain the current protocol message and extract the protocol name byte array. Calculate the current hash value based on the protocol name byte array using a hash algorithm. The query information is obtained by querying the mapping structure based on the current hash value. If the query results are a match, the corresponding protocol name string is obtained based on the match information, thus completing the protocol name resolution.
[0007] Furthermore, the method also includes: If the query results in a no-match, a corresponding protocol name string is created based on the current hash value and cached.
[0008] Furthermore, in response to the system startup command, the pre-registered set of protocol names is obtained, including: In response to the system startup command, all protocol message classes within the preset package path are obtained based on class scanning. Extract the class name identifier from each protocol message class to obtain the protocol name and put it into the pre-registered protocol name set.
[0009] Furthermore, the method also includes: Iterate through the set of pre-registered protocol names, calculate the hash value of each protocol name, and write the hash value into a temporary deduplication set; During the writing process, it is determined whether the current hash value exists in the temporary deduplication set; If so, then a hash collision information will be obtained; If not, then no hash collision information is obtained.
[0010] Furthermore, the method also includes: If no hash collision information is obtained, a mapping structure based on the primitive type key is used to store the hash value as the key and the protocol name string as the value.
[0011] Furthermore, the method also includes: If a hash collision occurs, an object key-based mapping structure is used, with the encapsulated protocol encoding object as the key and the protocol name string as the value for storage.
[0012] Furthermore, the encapsulated protocol encoding object includes: a 64-bit primary hash value, a 32-bit secondary hash value, and a reference to the original byte array.
[0013] Furthermore, by querying the mapping structure based on the current hash value, the query information is obtained, including: Hash location is performed based on the 32-bit auxiliary hash value of the current hash value to obtain a set of candidate protocol encoding objects; Compare the 64-bit master hash values of each candidate protocol encoding object to determine if they are the same; If so, the matching result of the current candidate protocol encoding object is obtained; Based on the matching results of the current candidate protocol encoding objects, compare the protocol name byte arrays to determine if they are the same; If so, the corresponding protocol name string will be returned, providing the query information.
[0014] Furthermore, the method also includes: During the lookup of the protocol routing table, "==" is used to determine the equivalence of references.
[0015] Furthermore, the protocol routing table can be a hash table, an array table, a linked list, or a tree-structured mapping table.
[0016] Furthermore, the mapping structure based on primitive type keys is ImmutableLongObjectMap.
[0017] Furthermore, the keys and values are stored in contiguous blocks of memory.
[0018] Furthermore, the hash algorithm is the FarmHash64 algorithm.
[0019] Furthermore, the capacity of the resource space pool is 256.
[0020] Secondly, embodiments of this application provide a protocol name resolution device based on hash mapping, the device comprising: The protocol acquisition module is used to obtain a set of pre-registered protocol names in response to the system startup command; The hash calculation module is used to put the byte arrays of each protocol name into the resource space pool, and calculate the hash value based on the hash algorithm. The mapping establishment module is used to establish a mapping structure from hash values to protocol name strings and to cache protocol name strings; The array calculation module is used to obtain the current protocol message and extract the protocol name byte array, calculate the current hash value based on the protocol name byte array using a hash algorithm; The mapping query module is used to query the mapping structure based on the current hash value and obtain query information; The protocol parsing module is used to obtain the corresponding protocol name string based on the query information when the query information is a match, thus completing the protocol name parsing.
[0021] Furthermore, the protocol acquisition module is specifically used to: in response to the system startup command, acquire all protocol message classes within the preset program package path based on class scanning; extract the class name identifier of each protocol message class to obtain the protocol name and put it into the pre-registered protocol name set.
[0022] Furthermore, the hash algorithm is the FarmHash64 algorithm.
[0023] Furthermore, the capacity of the resource space pool is 256.
[0024] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the steps of a protocol name resolution method based on hash mapping as described in any of the above embodiments.
[0025] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of a protocol name resolution method based on hash mapping as described in any of the above embodiments.
[0026] In summary, compared with the prior art, the beneficial effects of the technical solution provided in this application include at least the following: This application provides a protocol name resolution method based on hash mapping, which avoids the creation of temporary objects during protocol name resolution, reduces frequent memory allocation and release operations, and lowers runtime resource consumption. This application pre-registers a set of protocol names and stores the byte arrays of each protocol name in a resource space pool. It then calculates the corresponding hash value using a hash algorithm and constructs a mapping structure between the hash value and the protocol name string. Simultaneously, it caches the protocol name string. When obtaining a protocol message and resolving the protocol name, it can directly query the mapping structure based on the hash value to complete the protocol name matching, eliminating the need to repeatedly generate protocol name-related data objects and improving the processing speed of protocol name resolution. The use of hash algorithms ensures the accuracy and reliability of protocol name matching results, avoiding data processing errors caused by the creation of temporary objects. The cached protocol name string can be reused multiple times, reducing repetitive data calculation and processing steps and optimizing runtime efficiency. The centralized storage of protocol name byte arrays in the resource space pool standardizes the management of protocol name-related data, avoids system resource consumption caused by scattered data storage, makes the entire protocol name resolution process more streamlined and smooth, and improves the stability of long-term operation. Attached Figure Description
[0027] Figure 1 A flowchart illustrating a protocol name resolution method based on hash mapping, provided as an exemplary embodiment of this application.
[0028] Figure 2 A flowchart illustrating the steps of obtaining a set of pre-registered protocol names, provided as an exemplary embodiment of this application.
[0029] Figure 3 A flowchart of the steps for querying a mapping structure based on the current hash value, provided as an exemplary embodiment of this application.
[0030] Figure 4 This is a structural diagram of a protocol name resolution device based on hash mapping, provided as an exemplary embodiment of this application. Detailed Implementation
[0031] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0032] Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0033] Please see Figure 1This application provides a protocol name resolution method based on hash mapping, which specifically includes the following steps: Step S1: In response to the system startup command, obtain the set of pre-registered protocol names.
[0034] Please see Figure 2 In some embodiments, in response to a system startup command, a pre-registered set of protocol names is obtained, including: Step S11: In response to the system startup command, obtain all protocol message classes within the preset package path based on class scanning. Step S12: Extract the class name identifier for each protocol message class, obtain the protocol name, and put it into the pre-registered protocol name set.
[0035] In one feasible implementation, the pre-registered set of protocol names can be automatically collected by the system during the initialization phase.
[0036] Using the preset package path as the scan range can further save memory by limiting the scope.
[0037] Among them, the protocol message class is a concrete subclass of the message base class.
[0038] Specifically, when extracting class name identifiers for each protocol message class, it is preferable to extract the simple class name as the protocol name. The simple class name is the class name without the package name.
[0039] This application automatically acquires the pre-registered protocol name set upon system startup, eliminating the need for manual registration and effectively reducing manual operation costs. It avoids omissions and errors that may occur during manual registration, ensuring the completeness and accuracy of the pre-registered protocol name set. By using class scanning to obtain all protocol message classes within a preset package path, the scanning scope is limited, avoiding scanning irrelevant classes in the system, reducing system resource consumption, improving the efficiency of protocol message class acquisition, and accelerating the formation of the pre-registered protocol name set, laying an efficient foundation for subsequent protocol name resolution. Extracting simple class names from the protocol message classes as protocol names makes the protocol names more concise and consistent, facilitating subsequent protocol name identification and resolution. As a concrete subclass of the message base class, the protocol message class ensures that all extracted protocol names conform to a unified standard, avoiding resolution anomalies caused by inconsistent protocol names and improving the smoothness of the protocol name resolution process. The system automatically collects a set of pre-registered protocol names during the initialization phase, automating protocol name registration, reducing system maintenance difficulty, and adapting to the expansion requirements of protocol message classes. When a new protocol message class is added, no additional modifications to the registration process are required; the simple class name of the new protocol message class can be automatically included in the pre-registered protocol name set through class scanning, thus improving the overall operational stability of the protocol name resolution system.
[0040] In one feasible implementation, after collecting protocol names, a mapping relationship between protocol names and protocol message classes and protocol parsers is further constructed, and the set of protocol names is passed to the protocol name fast parsing module to establish a pre-computed protocol name mapping structure. This solution does not specifically limit the method for obtaining the aforementioned protocol names.
[0041] Step S2: Place the byte arrays of each protocol name into the resource space pool, and calculate the hash value based on the hash algorithm.
[0042] In some embodiments, the hash algorithm is the FarmHash64 algorithm.
[0043] In one feasible implementation, FarmHash64 can be replaced by other high-performance 64-bit hash algorithms such as MurmurHash3_64 and XXHash64, as long as the distribution characteristics and computational performance are comparable.
[0044] In some embodiments, the capacity of the resource space pool is 256.
[0045] By storing the byte array corresponding to the protocol name in a resource space pool and then calculating the hash value using a hash algorithm, the protocol identification data can be centrally organized and processed in an orderly manner, providing reliable support for rapid data location and efficient retrieval. Using the FarmHash64 algorithm for hash calculation, or replacing it with similar 64-bit hash algorithms such as MurmurHash3_64 and XXHash64, can quickly generate hash values based on the algorithm's computational advantages, ensuring a stable distribution of the generated hash values, avoiding hash collisions during data processing, and guaranteeing the accuracy of protocol data identification and differentiation.
[0046] Furthermore, setting the resource space pool capacity to 256 fully adapts to the actual usage needs of the protocol data processing scenario in this application. This capacity setting provides sufficient storage area for byte arrays with different protocol names, meeting the requirements of storing multiple types of protocol data simultaneously, while also reasonably controlling memory resource consumption and avoiding resource idleness and waste caused by excessive storage space. The adapted capacity configuration simplifies the scheduling and management of the resource space pool, improves the coordination between byte array storage and hash calculation processes, makes the overall data processing smoother, conforms to the system's normal operating logic, reduces the time required for resource allocation, further improves the overall efficiency of protocol data processing, and ensures stable and reliable operation.
[0047] Step S3: Establish a mapping structure from hash values to protocol name strings, and cache the protocol name strings.
[0048] By establishing a mapping structure between hash values and protocol name strings and caching the protocol name strings, the corresponding protocol name strings can be quickly matched directly based on the hash values. This avoids repeated retrieval and parsing operations of protocol name strings, reduces resource consumption caused by repeated storage of protocol name strings, lowers the memory usage of the system during runtime, speeds up the retrieval and recognition of protocol name strings, optimizes the processing flow of protocol-related data, improves the overall efficiency of the system in processing protocol information, shortens the time spent on data matching and retrieval, and maintains a high-efficiency operating state when processing various protocol-related businesses.
[0049] Step S4: Obtain the current protocol message and extract the protocol name byte array. Calculate the current hash value based on the protocol name byte array using a hash algorithm.
[0050] The hash algorithm used in step S4 is the same as the hash algorithm used in step S2.
[0051] The entire process does not require calling `new String()` and does not generate any temporary objects. It uses the same hash algorithm as the previous steps to calculate the protocol name byte array extracted from the current protocol message. This ensures that the generated hash value matches the hash value stored in the mapping structure, avoiding matching discrepancies caused by algorithm differences. The entire calculation process directly performs hash value calculations based on the protocol name byte array, eliminating the need for string conversion calls and preventing the generation of additional temporary objects. This reduces system resource consumption caused by the creation and destruction of temporary objects, simplifies the hash calculation process for protocol messages, accelerates the generation of the current hash value, improves the execution efficiency of protocol message data processing, and makes the matching and calculation process of various data more concise and efficient, reducing additional overhead in the system's data processing stage.
[0052] Step S5: Query the mapping structure based on the current hash value to obtain the query information.
[0053] Please see Figure 3 In some embodiments, the query information is obtained by querying the mapping structure based on the current hash value, including: Step S51: Perform hash positioning based on the 32-bit auxiliary hash value of the current hash value to obtain a set of candidate protocol encoding objects; Step S52: Compare the 64-bit master hash values of each candidate protocol encoding object to determine if they are the same; Step S53: If yes, then the matching result of the current candidate protocol encoding object is obtained; Step S54: Based on the matching results of the current candidate protocol encoding objects, compare the protocol name byte arrays to determine whether they are the same; Step S55: If yes, return the corresponding protocol name string to obtain the query information.
[0054] In one feasible implementation, the 32-bit auxiliary hash can employ different algorithms such as MurmurHash3_32 and CRC32C, with the main purpose of optimizing HashMap bucketing and reducing the probability of collisions.
[0055] In the process of retrieving query information by querying the mapping structure based on the current hash value, a 32-bit auxiliary hash value is first used for hash positioning. This quickly locks down the set of candidate protocol encoding objects, narrowing the query range and reducing the number of subsequent comparison operations, thereby speeding up the query and improving query efficiency. The 32-bit auxiliary hash can use algorithms such as MurmurHash3_32 and CRC32C, which can optimize the HashMap bucketing effect, reduce hash collisions, and avoid query process lag or efficiency degradation due to excessive collisions, ensuring the smoothness of the query process. After obtaining the set of candidate protocol encoding objects, a double verification is implemented by comparing the 64-bit main hash value of the candidate protocol encoding objects with the protocol name byte array in sequence. This accurately filters out the protocol encoding objects corresponding to the current hash value, avoiding mismatches and ensuring that the final query information has extremely high accuracy. The entire query process, through layered filtering and multi-verification design, balances query speed with the reliability of query results, improving system performance.
[0056] Step S6: If the query information is a match, the corresponding protocol name string is obtained based on the match information, and the protocol name parsing is completed.
[0057] By directly obtaining the corresponding protocol name string after determining the match of the query information, the overall process of protocol name parsing can be simplified, the data processing steps in the parsing process can be reduced, the execution speed of protocol name parsing can be improved, the accuracy and reliability of the protocol name parsing results can be ensured, the data error problem in the parsing process can be avoided, the execution logic of protocol name parsing can be made more concise and standardized, adapting to the protocol name parsing needs under different data query scenarios, and ensuring that the protocol name parsing operation is completed stably and orderly.
[0058] The steps described above ensure that each unique protocol name corresponds to a globally unique string instance. This application employs a pre-computed hash mapping mechanism, directly retrieving pre-cached string references at runtime through hash lookup, thus eliminating the creation of temporary string objects at runtime and overcoming the memory allocation pressure caused by frequent object creation. In PrimitiveMode, the protocol parsing process does not generate any heap memory allocation, achieving zero-allocation parsing. Compared to existing technologies that create a new String object for each parsing iteration, this solution eliminates object creation overhead in the normal runtime path.
[0059] The protocol name resolution method based on hash mapping provided in the above embodiments can avoid the creation of temporary objects during the protocol name resolution process, reduce frequent memory allocation and release operations in this stage, and reduce runtime resource consumption. This application pre-registers a set of protocol names and stores the byte arrays of each protocol name in a resource space pool. It then calculates the corresponding hash value using a hash algorithm and constructs a mapping structure between the hash value and the protocol name string. Simultaneously, it performs caching of the protocol name string. When obtaining a protocol message and resolving the protocol name, it can directly query the mapping structure based on the hash value to complete the protocol name matching, eliminating the need to repeatedly generate protocol name-related data objects and improving the processing speed of protocol name resolution. The use of hash algorithms ensures the accuracy and reliability of protocol name matching results, avoiding data processing errors caused by the creation of temporary objects. The cached protocol name string can be reused multiple times, reducing repetitive data calculation and processing steps and optimizing runtime efficiency. The centralized storage of protocol name byte arrays in the resource space pool standardizes the management of protocol name-related data, avoids system resource consumption caused by scattered data storage, makes the entire protocol name resolution process more streamlined and smooth, and improves the stability of long-term operation.
[0060] In some embodiments, the method further includes: If the query results in a no-match, a corresponding protocol name string is created based on the current hash value and cached.
[0061] When a query fails to find a match (i.e., when no protocol is registered), a fallback logic is executed: a protocol name string is dynamically created and cached for later reuse. Although an object is created in this path, it only occurs upon the first encounter with a new protocol and does not affect the zero-allocation characteristic of the regular path. This further eliminates runtime object creation, significantly reducing GC pressure and substantially improving system throughput.
[0062] In one feasible implementation, dynamic protocol registration can employ different thread safety strategies, such as read-write locks, segmented locks, or lock-free concurrent structures, depending on the specific concurrency scenario.
[0063] In one feasible implementation, the above scheme performs optimally when there are no collisions in the 64-bit hash space, but it does not consider the extreme scenario of hash collisions. To ensure the correctness of the system under any circumstances, an adaptive degradation protection mechanism is further added.
[0064] In some embodiments, the method further includes: Iterate through the set of pre-registered protocol names, calculate the hash value of each protocol name, and write the hash value into a temporary deduplication set; During the writing process, it is determined whether the current hash value exists in the temporary deduplication set; If so, then a hash collision information will be obtained; If not, then no hash collision information is obtained.
[0065] Specifically, by traversing the pre-registered protocol name set, calculating the hash value of each protocol name and writing it into a temporary deduplication set, and judging whether the current hash value exists in the temporary deduplication set during the writing process, it is possible to obtain hash conflict information or no hash conflict information in a timely manner, providing a reliable basis for triggering the adaptive degradation protection mechanism, further improving the stability of system operation, ensuring the smooth progress of protocol-related processing, and avoiding processing errors caused by hash conflicts.
[0066] In one feasible implementation, the pre-built mapping structure uses different storage media depending on the operating mode.
[0067] The mapping structure preferably uses a mapping container that supports primitive type keys to avoid boxing overhead caused by wrapper types, thereby improving query efficiency. Simultaneously, an immutable mapping structure is preferred to ensure thread safety. Using a mapping container that supports primitive type keys reduces additional overhead in data processing, accelerates data query execution, and improves the overall data processing efficiency of the system. Employing an immutable mapping structure ensures stable data interaction in a multi-threaded environment, avoids data anomalies caused by multi-threaded operations, and guarantees the stability and reliability of the system operation.
[0068] In some embodiments, the method further includes: If no hash collision information is obtained, a mapping structure based on the primitive type key is used to store the hash value as the key and the protocol name string as the value.
[0069] The use of a mapping structure based on primitive type keys simplifies data storage, reduces redundant processing steps, and improves the efficiency of storing and retrieving protocol name-related data. Using hash values as keys ensures the accuracy of data indexing, avoiding index deviations caused by hash collisions. Furthermore, the mapping structure based on primitive type keys adapts to actual data storage needs, making the storage state of protocol name strings more stable, accelerating data retrieval and matching, reducing resource consumption during data processing, and improving the overall smoothness of data interaction and parsing.
[0070] In some embodiments, the method further includes: If a hash collision occurs, an object key-based mapping structure is used, with the encapsulated protocol encoding object as the key and the protocol name string as the value for storage.
[0071] In some embodiments, the encapsulated protocol encoding object includes: a 64-bit primary hash value, a 32-bit secondary hash value, and a reference to the original byte array.
[0072] When a hash collision is detected, a key-based mapping structure is used for storage, effectively avoiding storage anomalies caused by hash collisions. This ensures the accurate and traceable correspondence between the encapsulated protocol encoding object and the protocol name string, guaranteeing the stability and accuracy of data storage. The encapsulated protocol encoding object integrates a 64-bit primary hash value, a 32-bit secondary hash value, and a reference to the original byte array. This allows for the complete preservation of key information related to the protocol encoding while achieving effective data storage. It provides ample information support for subsequent querying, verification, and related data processing operations, reducing information loss during data processing, improving processing efficiency, simplifying the handling process after a hash collision, and reducing the complexity of data storage and processing.
[0073] In some embodiments, the method further includes: During the lookup of the protocol routing table, "==" is used to determine the equivalence of references.
[0074] The aforementioned pre-caching mechanism ensures that the same protocol name always returns a reference to the same string object. When routing protocols, "==" can be used directly to check the reference equality, replacing the content comparison of equals().
[0075] In some embodiments, the protocol routing table is a hash table, an array table, a linked list, or a tree-structured mapping table.
[0076] In the protocol routing table, string references are used as keys (not string content). During route lookup, the string reference returned from the parser is directly compared with the key reference in the routing table using "==". This operation is a single CPU instruction with a time complexity of O(1). Specifically, the protocol routing table can be implemented using mapping containers such as hash tables, array tables, linked lists, or tree mapping tables; this application does not impose any restrictions on this. Building upon the elimination of object creation, the efficiency of protocol route comparison is further improved by n times (where n is the average length of the protocol name), resulting in a significant improvement in overall performance.
[0077] The pre-computation caching mechanism ensures that each unique protocol name corresponds to a globally unique string instance. Therefore, the reference equality check "==" can be used instead of the equals() method for protocol routing comparison, thus overcoming the problem of low efficiency in string comparison. All parsing operations return references to the same string instance, and the protocol routing process only needs to compare object addresses, reducing the time complexity from O(n) to O(1).
[0078] In some embodiments, the mapping structure based on primitive type keys is ImmutableLongObjectMap.
[0079] Furthermore, this application optimizes the memory efficiency of the storage structure.
[0080] In the standard Map structure, hash values of type long need to be boxed into long objects as keys, resulting in additional object header and reference overhead. This application can be improved by using EclipseCollections' ImmutableLongObjectMap, which uses the original long type directly as the key, avoiding long boxing and eliminating the memory overhead of key objects.
[0081] In one feasible implementation, ImmutableLongObjectMap can be replaced with other third-party library implementations that support primitive type keys, such as FastUtil's Long2ObjectOpenHashMap, or the JDK standard Long2ObjectMap (requires JDK 21+).
[0082] Specifically, by using pre-cached string instances and combining them with a primitive type Map to avoid the boxing overhead of key objects, the metadata waste of duplicate instances and the overhead of key object creation are eliminated, thus overcoming the problem of high memory consumption. Each unique protocol corresponds to only one string instance, and ImmutableLongObjectMap uses the primitive long type as the key, avoiding the object header overhead of the Long wrapper class.
[0083] In one feasible implementation, the pre-cached string instances are ensured to be immutable objects (modified by `final`, with the internal `char` array not exposed), allowing for safe sharing by multiple threads. Combined with the aforementioned reference comparison mechanism, the entire parsing and routing process eliminates the need to copy strings, further reducing memory fluctuations. This solution ensures a unique corresponding string instance for each protocol name through pre-registration and uses `intern` as an optional string pooling optimization technique. While improving performance, protocol-related memory usage is reduced by over 95%, and memory usage does not increase with QPS, achieving a qualitative leap in memory efficiency.
[0084] By using pre-cached string instances with fixed addresses, frequent garbage collection (GC) avoids object address changes, allowing the CPU to fully utilize the cache prefetching mechanism. The fixed storage address of pre-cached strings improves cache hit rate with repeated access.
[0085] In some embodiments, keys and values are stored in contiguous blocks of memory.
[0086] Furthermore, this application further optimizes the memory layout to improve cache hit rate.
[0087] The ImmutableLongObjectMap internally employs open addressing, storing keys (raw long arrays) and values (string reference arrays) in contiguous memory blocks. Compared to the JDK HashMap's scattered storage of linked list / red-black tree nodes, this contiguous memory layout better aligns with the CPU cache line prefetching mechanism. This contiguous storage structure improves spatial locality of data access by compactly arranging keys and values in array form, enabling more efficient utilization of the CPU cache line prefetching mechanism during sequential access, thereby reducing the performance overhead caused by cache misses. In one specific implementation, the CPU cache line size is typically tens of bytes (e.g., 64 bytes), but this application does not depend on a specific cache line size.
[0088] Based on the aforementioned pre-caching mechanism, the address of string instances remains fixed throughout the system's lifecycle. Further reference comparisons and primitive type Maps avoid creating new objects at runtime, eliminating cache invalidation caused by object address changes due to garbage collection. Repeated access to the same batch of protocol strings at fixed addresses allows the CPU cache to be fully warmed up, significantly improving the hit rate.
[0089] Furthermore, the data structures accessed by the runtime parsing process (calculating hashes → querying the Map → returning references) (byte arrays, hash values, Map internal arrays, string references) exhibit good locality in memory, reducing memory access latency caused by cache misses. Building upon the above optimizations, the CPU cache hit rate is further improved, memory access latency is reduced, and overall performance approaches its theoretical limit.
[0090] In one feasible implementation, when a collision occurs, a protocol code object (ProtocolCode) is used as the Map key. This object contains three layers of verification data: a 64-bit primary hash value (FarmHash64), a 32-bit secondary hash value (MurmurHash3_32), and a reference to the original byte array. A layered matching mechanism is used during the search: First, the mapping container uses the 32-bit secondary hash value to perform hashing and locate the candidate protocol code object set. Second, during the comparison of candidate objects, the 64-bit primary hash value is compared first. If they do not match, the current candidate object is determined to be mismatched, and other candidate objects are compared, thus quickly filtering out 99.99% of mismatches and reducing byte comparisons. If they match, the process proceeds to the next step. Third, the contents of the original byte array are compared. If they are completely identical, a successful match is determined, and the corresponding protocol name string instance is returned. If all candidate objects fail to match, it is determined to be a miss, and a fallback process is initiated, converting the protocol name byte array into a string object and returning it to ensure that unregistered protocols can still be parsed and processed at runtime.
[0091] The mapping mode based on primitive type keys is called PrimitiveMode, while the mapping mode based on object keys is called ObjectiveMode. In conflict scenarios, ObjectiveMode needs to create temporary ProtocolCode objects (performance decrease of 30-40%), but it is still significantly better than traditional solutions. The automatic degradation mechanism further ensures that the system can achieve a balance between correctness and performance without manual intervention. When registering a new protocol at runtime through fallback logic, it automatically detects whether a new conflict has been introduced. If the new protocol causes a conflict, the system automatically switches from PrimitiveMode to ObjectiveMode, rebuilds the mapping structure, and ensures that correctness is not affected. Based on performance optimization, fault tolerance for extreme scenarios has been added. The system achieves theoretically optimal performance when there is no conflict, and automatically degrades to ensure correctness in the event of a conflict, thus achieving a balance between performance and stability.
[0092] In one feasible implementation, in addition to automatic degradation, manual intervention methods such as conflict protocol renaming and conflict detection alarms can also be used. Although the degree of automation is sacrificed to some extent, conflict resolution can be achieved quickly.
[0093] Please see Figure 4 Another embodiment of this application provides a protocol name resolution device based on hash mapping, the device comprising: The protocol acquisition module 101 is used to obtain a set of pre-registered protocol names in response to the system startup command.
[0094] The hash calculation module 102 is used to put the byte arrays of each protocol name into the resource space pool, and calculate the hash value based on the hash algorithm.
[0095] The mapping establishment module 103 is used to establish a mapping structure from hash values to protocol name strings and to cache protocol name strings.
[0096] The array calculation module 104 is used to obtain the current protocol message and extract the protocol name byte array, calculate the protocol name byte array based on the hash algorithm, and obtain the current hash value.
[0097] The mapping query module 105 is used to query the mapping structure based on the current hash value and obtain query information.
[0098] The protocol parsing module 106 is used to obtain the corresponding protocol name string based on the hit information when the query information is a hit information, and to complete the protocol name parsing.
[0099] In some embodiments, the protocol acquisition module is specifically used to: in response to a system startup command, acquire all protocol message classes within a preset package path based on a class scanning method; extract the class name identifier for each protocol message class to obtain the protocol name and place it into a pre-registered protocol name set.
[0100] In some embodiments, the hash algorithm is the FarmHash64 algorithm.
[0101] In some embodiments, the capacity of the resource space pool is 256.
[0102] The specific limitations of the protocol name resolution device based on hash mapping provided in this embodiment can be found in the embodiment of the protocol name resolution method based on hash mapping described above, and will not be repeated here. Each module in the above-described protocol name resolution device based on hash mapping can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0103] This application provides a computer device that may include a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it causes the processor to perform steps of a hash-based protocol name resolution method as described in any of the above embodiments.
[0104] The working process, working details, and technical effects of the computer device provided in this embodiment can be found in the embodiment of a protocol name resolution method based on hash mapping described above, and will not be repeated here.
[0105] This application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of a protocol name resolution method based on hash mapping as described in any of the above embodiments. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or Memory Sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
[0106] The working process, working details, and technical effects of the computer-readable storage medium provided in this embodiment can be found in the embodiment of a protocol name resolution method based on hash mapping described above, and will not be repeated here.
[0107] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0108] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0109] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A protocol name resolution method based on hash mapping, characterized in that, The method includes: In response to the system startup command, obtain the set of pre-registered protocol names; The byte arrays of each protocol name are placed into the resource space pool, and the hash value is obtained by calculating the hash value of the byte arrays based on the hash algorithm. Establish a mapping structure from hash values to protocol name strings, and cache the protocol name strings; Obtain the current protocol message and extract the protocol name byte array. Calculate the current hash value based on the protocol name byte array using a hash algorithm. The mapping structure is queried based on the current hash value to obtain query information; If the query information is a match, the corresponding protocol name string is obtained based on the match information, and the protocol name parsing is completed.
2. The protocol name resolution method based on hash mapping according to claim 1, characterized in that, Also includes: If the query results in a no-match, a corresponding protocol name string is created based on the current hash value and cached.
3. The protocol name resolution method based on hash mapping according to claim 1, characterized in that, The step of obtaining the pre-registered protocol name set in response to the system startup command includes: In response to the system startup command, all protocol message classes within the preset package path are obtained based on class scanning. Extract the class name identifier from each protocol message class to obtain the protocol name and put it into the pre-registered protocol name set.
4. The protocol name resolution method based on hash mapping according to claim 1, characterized in that, Also includes: Traverse the set of pre-registered protocol names, calculate the hash value of each protocol name, and write the hash value into a temporary deduplication set; During the writing process, it is determined whether the current hash value exists in the temporary deduplication set; If so, then a hash collision information will be obtained; If not, then no hash collision information is obtained.
5. The protocol name resolution method based on hash mapping according to claim 4, characterized in that, Also includes: If no hash collision information is obtained, a mapping structure based on the primitive type key is used to store the hash value as the key and the protocol name string as the value.
6. The protocol name resolution method based on hash mapping according to claim 4, characterized in that, Also includes: If a hash collision occurs, an object key-based mapping structure is used, with the encapsulated protocol encoding object as the key and the protocol name string as the value for storage.
7. The protocol name resolution method based on hash mapping according to claim 6, characterized in that, The encapsulated protocol encoding object includes: a 64-bit main hash value, a 32-bit auxiliary hash value, and a reference to the original byte array.
8. The protocol name resolution method based on hash mapping according to claim 7, characterized in that, The step of querying the mapping structure based on the current hash value to obtain query information includes: Hash location is performed based on the 32-bit auxiliary hash value of the current hash value to obtain a set of candidate protocol encoding objects; Compare the 64-bit master hash values of each candidate protocol encoding object to determine if they are the same; If so, the matching result of the current candidate protocol encoding object is obtained; Based on the matching results of the current candidate protocol encoding objects, compare the protocol name byte arrays to determine whether they are the same; If so, the corresponding protocol name string will be returned, providing the query information.
9. The protocol name resolution method based on hash mapping according to claim 1, characterized in that, Also includes: During the lookup of the protocol routing table, "==" is used to determine the equivalence of references.
10. The protocol name resolution method based on hash mapping according to claim 9, characterized in that, The protocol routing table can be a hash table, an array table, a linked list, or a tree-structured mapping table.
11. The protocol name resolution method based on hash mapping according to claim 5, characterized in that, The mapping structure based on primitive type keys is ImmutableLongObjectMap.
12. The protocol name resolution method based on hash mapping according to claim 11, characterized in that, The key and the value are stored in contiguous blocks of memory.
13. The protocol name resolution method based on hash mapping according to claim 1, characterized in that, The hash algorithm is the FarmHash64 algorithm.
14. The protocol name resolution method based on hash mapping according to claim 1, characterized in that, The capacity of the resource space pool is 256.
15. A protocol name resolution device based on hash mapping, characterized in that, The device includes: The protocol acquisition module is used to obtain a set of pre-registered protocol names in response to the system startup command; The hash calculation module is used to put the byte arrays of each protocol name into the resource space pool, and calculate the hash value based on the hash algorithm. The mapping establishment module is used to establish a mapping structure from hash values to protocol name strings and to cache the protocol name strings; The array calculation module is used to obtain the current protocol message and extract the protocol name byte array, and calculate the current hash value based on the protocol name byte array using a hash algorithm; The mapping query module is used to query the mapping structure based on the current hash value to obtain query information. The protocol parsing module is used to obtain the corresponding protocol name string based on the hit information when the query information is a hit information, and to complete the protocol name parsing.
16. The protocol name resolution device based on hash mapping according to claim 15, characterized in that, The protocol acquisition module is specifically used to: in response to the system startup command, acquire all protocol message classes in the preset program package path based on class scanning; extract the class name identifier of each protocol message class to obtain the protocol name and put it into the pre-registered protocol name set.
17. The protocol name resolution device based on hash mapping according to claim 15, characterized in that, The hash algorithm is the FarmHash64 algorithm.
18. The protocol name resolution device based on hash mapping according to claim 15, characterized in that, The capacity of the resource space pool is 256.
19. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the protocol name resolution method based on hash mapping as described in any one of claims 1 to 14.
20. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the protocol name resolution method based on hash mapping as described in any one of claims 1 to 14.