Distributed employee attribute real-time screening method and system based on compact coding

Abstract

The application discloses a distributed employee attribute real-time screening method and system based on compact coding and belongs to the technical field of computer data processing. The method comprises the following steps: establishing a global enumeration dictionary for screenable employee attributes and distributing variable-length digital identifiers; encoding each employee attribute into a compact attribute feature code comprising a header check area, a hierarchical identifier area and a main attribute coding area to form a component of employee main data; distributing a key-value pair of an employee ID and the attribute feature code to the local memory of each business application server through a data synchronization mechanism to form a distributed employee feature code set; analyzing screening conditions in a screening request and generating a screening mask equal in length to the attribute feature code by using the global enumeration dictionary; and performing parallel traversal of the feature code set in the local memory of each business application server, performing segmented comparison and bit-level matching operation on the attribute feature code and the screening mask, and obtaining an employee ID that is completely matched as a real-time screening result.

Description

Technical Field

[0001] This invention relates to the field of computer data processing technology, specifically to a distributed real-time screening method and system for employee attributes based on compact coding. Background Technology

[0002] In human resource management systems, real-time filtering of employee attribute information is a high-frequency, fundamental operation in numerous business scenarios, including organizational management, talent inventory, and emergency communication. Employee attributes typically involve dozens of dimensions, such as department, job level, job sequence, skill tags, contract type, and educational background. Faced with tens or even hundreds of thousands of employees, it is necessary to quickly find sets of employees matching multiple conditions from massive amounts of data. Existing technologies mostly employ centralized relational databases or search engines to implement employee attribute filtering. Relational database solutions distribute attributes across multiple fields, and multi-condition filtering relies on structured query languages ​​for row-by-row scanning or index merging. When the attribute dimensions increase and the filtering conditions become complex, queries need to perform multi-column matching and table join operations on large amounts of data, often resulting in response times of several seconds. Search engine solutions accelerate text search by building inverted indexes, but for precise attribute matching scenarios, multiple set intersection and merge operations are still required, leading to high index memory overhead and limited scalability. Both solutions are limited by centralized architectures, unable to fully utilize distributed computing resources, and struggle to stably provide millisecond-level real-time filtering capabilities under high concurrency. The redundancy in the encoding of attribute information during storage and matching, along with the inefficient bit-level comparison method, prevents further reduction in the computational overhead of a single attribute matching. At the same time, the centralized attribute storage and filtering architecture hinders the parallel execution of high-speed memory-level computations on distributed service nodes, making it impossible to achieve low-latency responses to large-scale real-time filtering requests. Summary of the Invention

[0003] This paper presents a distributed real-time employee attribute filtering method and system based on compact coding. The method compresses multi-dimensional employee attributes into compact feature codes and stores them in the local memory of each server. By using filtering masks and parallel bit-level matching operations, the method reduces the attribute data matching latency and improves the real-time performance of large-scale employee attribute filtering.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a distributed real-time filtering method for employee attributes based on compact coding, comprising: establishing a global enumeration dictionary for all filterable employee attributes in the human resource management system; assigning a variable-length digital identifier to each attribute value, so that high-frequency attribute values ​​can occupy a shorter coding width, thereby compressing the storage space of attribute information as a whole; encoding all attribute values ​​of each employee into a compact attribute feature code containing a header verification area, a hierarchical identifier area, and a main attribute coding area according to the digital identifier, and using the attribute feature code as a component of the employee master data, using a single compact code to replace multi-field storage, effectively reducing memory overhead; forming a key-value pair between the employee ID and the corresponding attribute feature code, and distributing it to the local memory of all business application servers through a data synchronization mechanism. This process forms a distributed set of employee signature codes, enabling each business node to hold a complete and real-time updated copy of the employee signature codes locally. When an attribute filtering request is received from a client, the filtering conditions contained in the request are parsed, and the filtering conditions are converted into a filtering mask of the same length as the attribute signature codes according to the global enumeration dictionary, allowing multiple attribute combinations to be expressed at once. Each business application server traverses the distributed set of employee signature codes in parallel in its local memory, performing segmented comparisons and bit-level matching operations between the attribute signature codes corresponding to each employee ID and the filtering mask, filtering out all employee IDs whose attribute signature codes completely match the filtering mask as the real-time filtering results. This allows for memory-level parallel computation to be completed locally on the business application server, avoiding remote database queries and cross-network data transmission, and achieving millisecond-level real-time filtering response.

[0005] As a technical solution of the present invention, when establishing a global enumeration dictionary, the filterable employee attributes are divided according to attribute categories and a category identifier field is assigned to each attribute category. For each specific attribute value under the same attribute category, the length of the numerical identifier of each attribute value is determined according to the estimated number of all possible values ​​within the attribute category, thereby flexibly adopting differentiated bit widths for different attribute categories and further reducing coding redundancy. The global enumeration dictionary is generated by summarizing the category identifier fields of all attribute categories and the numerical identifiers corresponding to each attribute value, and the starting position and occupancy length of each attribute category in the attribute feature code are recorded, providing a deterministic basis for bit segment division for encoding and decoding.

[0006] As another technical solution of the present invention, when generating compact attribute feature codes, the encoding scheme version number and the overall verification information of the attribute feature codes are filled into a fixed number of bits in the header verification area to support backward compatibility and integrity verification of the encoding format; the layered attribute values ​​of each employee under the preset employee layered dimensions are obtained and converted into layered identifier bits in the corresponding positions in the layered identifier area. The layered dimensions include personnel category dimension, core department affiliation dimension, and emergency contact status dimension. The layered identifier area is used to achieve coarse-grained and rapid exclusion; according to the actual attribute values ​​of each employee under each attribute category, the corresponding numerical identifiers are obtained from the global enumeration dictionary, and the numerical identifiers are sequentially concatenated to the main attribute encoding area according to the starting position and placeholder length of the record. For employees who have no actual attribute values ​​under a certain attribute category, a preset empty value code is filled in the corresponding position in the main attribute encoding area to ensure that the feature code is of fixed length to facilitate hardware alignment acceleration.

[0007] In a preferred embodiment of the present invention, when forming a distributed employee feature code set, if the employee attribute information in the core human resource management system changes, the employee's attribute feature code is recalculated to generate an updated attribute feature code. The employee ID and the updated attribute feature code are assembled into a data synchronization message and broadcast to all business application servers through an enterprise-level message bus. After receiving the data synchronization message, each business application server searches for and replaces the corresponding attribute feature code in the global employee feature code mapping table maintained in local memory according to the employee ID, thus completing the real-time update of the distributed employee feature code set. This ensures the consistency of the data used for screening by each business node and shortens the delay from master data changes to searchability.

[0008] As a specific implementation of the present invention, when converting the filter mask, the attribute filtering request is parsed to extract all relevant filter attribute categories and their corresponding filter attribute values. Based on the starting position and placeholder length of each attribute category recorded in the global enumeration dictionary, a corresponding mask bit segment is allocated in the filter mask for each filter attribute category. For specified filter attribute categories, the numeric identifier corresponding to the filter attribute value is filled into the precise bit of the mask bit segment. For unspecified attribute categories, wildcard bits are filled into the mask bit segment. When the attribute filtering request contains a filtering instruction for employee stratification, a stratification precision value is also set at the position of the corresponding stratification identifier bit in the filter mask, so that the filter mask can simultaneously carry multi-level filtering semantics and has the same length as the attribute feature code, which facilitates direct bit-level parallel matching.

[0009] As a further improvement of the present invention, during the traversal and comparison process, each business application server divides the distributed employee feature code set in its local memory into multiple data segments and allocates an independent scanning thread to each data segment, enabling multiple scanning threads to execute the traversal task in parallel. Each scanning thread sequentially reads the attribute feature code corresponding to each employee ID in its allocated data segment, extracts the hierarchical identifier segment in the attribute feature code and performs an initial comparison with the corresponding hierarchical identifier segment in the filter mask. When the hierarchical identifier segments are not completely consistent, the current employee ID is skipped directly, quickly filtering out a large number of irrelevant employees. When the hierarchical identifier segments are completely consistent, the scanning thread continues to perform segment-by-segment bit-level comparison of the main attribute encoding area of ​​the attribute feature code with the corresponding mask bit segment in the filter mask until the comparison of all bit segments is completed or a mismatched bit segment is encountered, at which point the comparison stops. By combining hierarchical pre-filtering with segment-by-segment precise matching, the average comparison overhead is significantly reduced, and the throughput of high-concurrency filtering is improved.

[0010] Preferably, in the segment-by-segment bit-level comparison, the scanning thread extracts the first encoded segment corresponding to the current attribute category from the attribute feature code and the corresponding second mask segment from the filter mask according to the arrangement order of each attribute category in the main attribute encoding area. It then determines whether each bit of the second mask segment is a wildcard bit. If so, the current attribute category is confirmed to match and the process continues to the next attribute category. If it is not a wildcard bit, a complete consistency comparison is performed between each bit of the first encoded segment and each bit of the second encoded segment. If all bits are the same, the current attribute category is confirmed to match; otherwise, the matching fails and the comparison operation for the current employee ID is terminated. This mechanism ensures early termination if any bit does not match, avoiding invalid complete comparisons and further accelerating the filtering speed.

[0011] When allocating scanning threads, each application server obtains the number of cores in its local central processing unit and determines the initial number of scanning threads based on this number. It also obtains the total number of employee IDs in the distributed employee feature code set and, combined with the initial number of scanning threads, calculates the number of employee feature codes in each data segment, dividing the data segments sequentially according to storage order. An independent scanning thread is allocated to each data segment, and the start and end employee ID positions are passed to the corresponding thread. After completing its own data segment traversal, each scanning thread retrieves a new data segment from other data segments that have not yet been traversed and continues execution until all data segments have been traversed, achieving dynamic load balancing and fully utilizing multi-core computing resources. The wildcard bit is filled in the filter mask with a special binary value that is different from all valid numeric identifiers to ensure the reliability of the matching logic and the safety of bitwise operations.

[0012] This invention also provides a distributed real-time employee attribute screening system based on compact coding, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the aforementioned method. This system compresses multi-dimensional employee attribute information through compact attribute feature codes, utilizes local memory distributed storage and multi-threaded parallel bit-level matching, and directly completes high-concurrency real-time employee attribute screening on the business application server side without relying on centralized database computation. It balances response speed and resource consumption, making it suitable for real-time human resource scheduling and decision-making scenarios in large-scale organizations.

[0013] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0014] The human resource management system categorizes filterable employee attributes and establishes a global enumeration dictionary. For specific attribute values ​​within the same category, variable-length numeric identifiers are assigned based on the estimated number of possible values, ensuring the placeholder length matches the value range and avoiding wasted space due to fixed lengths. The category identifiers and numeric identifiers record the starting position and placeholder length of each attribute category in the attribute feature code, generating a compact attribute feature code for each employee, including a header verification area, a hierarchical identifier area, and a main attribute encoding area. The hierarchical identifier area encodes hierarchical dimensions such as personnel category, core department affiliation, and emergency contact status into a few hierarchical identifier bits. The main attribute encoding area concatenates the corresponding numeric identifiers according to the order of each attribute category, filling empty attributes with null values. This encoding method transforms scattered multidimensional text attributes into binary codes with a fixed total length and segmented structure, eliminating redundant string storage overhead. Employee attributes are converted into compact data units capable of bit-level operations, allowing the matching process to be completed directly at the processor bit level. The key-value pairs composed of employee IDs and attribute feature codes are distributed to the local memory of each business application server through a data synchronization mechanism, forming a distributed employee feature code set. Upon receiving an attribute filtering request, the relevant attribute categories and values ​​are parsed. A filtering mask of the same length as the attribute feature code is generated based on the starting position and placeholder length of the global enumeration dictionary record. For attribute categories with specified filtering values, precise numerical identifiers are filled into the corresponding mask bit segments; for unspecified attribute categories, wildcard bits are filled in. Layered filtering conditions are converted into layered precise values. Each application server uses multi-threading to traverse the local feature code set in parallel, first extracting the layered identifier segment of the attribute feature code and comparing it with the corresponding segment of the filtering mask, quickly skipping a large number of mismatched records. Records with consistent layering are then compared bit-by-bit with the mask bit segments segment by segment according to the main attribute encoding area. If the mask bit segment is a wildcard, the precise comparison of that attribute category is skipped; if it is not a wildcard, a completely consistent bit match is performed. If any bit is inconsistent, the subsequent comparison of that record is terminated. All operations are performed in the server's local memory, without disk read / write or cross-network data transfer. Bit-level comparisons are performed directly by the central processing unit. The segmented matching and early termination mechanisms significantly reduce the average number of comparisons per record, enabling parallel real-time filtering of large-scale employee attributes in a distributed environment. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0016] Figure 1 This is a flowchart of a distributed real-time employee attribute filtering method based on compact coding; Figure 2 This is a flowchart of the employee attribute feature code encoding method; Figure 3 This is a flowchart illustrating the process of synchronously updating the feature code after changes in employee attribute information. Figure 4 This is a flowchart of the filter mask generation process; Figure 5 This is a flowchart of a distributed feature code parallel comparison process based on dynamic task allocation. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 This invention provides a distributed real-time employee attribute filtering method based on compact coding, comprising the following steps: During implementation, a global enumeration dictionary is established for all filterable employee attributes in the human resource management system, and a variable-length numerical identifier is assigned to each attribute value. Each attribute and attribute value participating in the filtering process obtains a unique and variable-length numerical representation in this dictionary. Based on this numerical identifier, all attribute values ​​of each employee are encoded into a compact attribute feature code. This attribute feature code comprises three components: a header verification area, a hierarchical identifier area, and a main attribute encoding area. This attribute feature code is stored and maintained as a component of the employee's master data. Each employee ID and its corresponding attribute feature code are combined into key-value pairs, which are distributed to the local memory of all business application servers through a data synchronization mechanism, forming a complete distributed employee feature code set on each server. When any business application server receives an attribute filtering request from a client, it parses the filtering conditions contained in the request and converts these conditions into a filtering mask of the same length as the attribute feature code based on the global enumeration dictionary. Each business application server traverses the distributed employee feature code set in parallel in its local memory, performs segmented comparison and bit-level matching operations on the attribute feature code corresponding to each employee ID with the filter mask, filters out all employee IDs whose attribute feature codes completely match the filter mask, and summarizes the employee IDs that meet the conditions into real-time filtering results and returns them.

[0019] Example 1: In specific implementation, please refer to Figure 2For all filterable employee attributes in the human resource management system, they are categorized according to attribute types, and a category identifier field is assigned to each attribute type. Attribute type refers to the classification field in employee information that can be used as a filtering condition, such as education level, work location, job level, etc. A unique category identifier field is assigned to each attribute type; this category identifier field serves as a unique coded prefix for the attribute type and is used to distinguish different attribute types in the coding structure.

[0020] When determining the length of the numeric identifier for each specific attribute value within each attribute category, first estimate the number of all possible values ​​within that attribute category. This estimate is obtained by collecting all historical attribute values ​​that have appeared under that attribute category in the human resource management system, and combining this with potential new attribute values ​​from the business expansion plan to calculate an upper limit value. The minimum number of bits required to represent this upper limit value in binary is then used to determine the length of the numeric identifier for each attribute value within that attribute category. This ensures that each attribute value receives a fixed-length numeric identifier within that attribute category, and that the numeric identifiers for any two different attribute values ​​are distinct. For attribute categories with a smaller number of possible values, the numeric identifier length is shorter; for attribute categories with a larger number of possible values, the numeric identifier length is longer.

[0021] During the generation of the global enumeration dictionary, the category identifier fields of all attribute categories are summarized, along with all attribute values ​​and their corresponding numeric identifiers under each attribute category. In the global enumeration dictionary, a starting position and a placeholder length are recorded for each attribute category. The starting position of an attribute category indicates the position of the first bit of its numeric identifier in the main attribute encoding area; the placeholder length of an attribute category is equal to the length of the numeric identifier within that category, i.e., the number of bits occupied by the numeric identifier corresponding to that attribute category. The order of attribute categories in the global enumeration dictionary determines their order in the main attribute encoding area, calculated by sequentially adding the placeholder length of the previous attribute category to the starting positions of each attribute category according to their order of arrangement.

[0022] After obtaining the global enumeration dictionary, the employee attribute information is encoded. The encoding scheme version number and the overall verification information of the attribute feature code are filled into a fixed number of bits in the header verification area of ​​the attribute feature code. The encoding scheme version number is represented by a binary field of a preset length, used to identify the currently used encoding structure version for compatibility handling during subsequent encoding upgrades. The overall verification information is the integrity verification value of the remaining part of the attribute feature code excluding the header verification area. Optionally, it is calculated using a cyclic redundancy check algorithm, occupying a fixed number of bits and placed at the end of the header verification area.

[0023] When filling the hierarchical identifier area, the preset employee hierarchical dimensions include personnel category, core department affiliation, and emergency contact status. For each employee, the actual attribute values ​​under the personnel category dimension (e.g., full-time employee, outsourced employee, intern) are extracted from the human resources management system; the actual attribute values ​​under the core department affiliation dimension (e.g., whether they belong to a core R&D department) are extracted; and the actual attribute values ​​under the emergency contact status dimension (e.g., whether they are an emergency contact person) are extracted. These actual attribute values ​​under these three hierarchical dimensions are mapped to preset binary code values ​​as hierarchical identifier bits. The hierarchical identifier bits in the hierarchical identifier area are arranged sequentially according to the personnel category dimension, core department affiliation dimension, and emergency contact status dimension. Each hierarchical identifier bit occupies a fixed number of bits, determined by the number of possible values ​​for each hierarchical dimension.

[0024] For each employee's actual attribute value under each attribute category, the corresponding numeric identifier for that attribute value under that category is searched in the global enumeration dictionary. Following the starting position and placeholder length of each attribute category recorded in the global enumeration dictionary, the found numeric identifiers are sequentially concatenated and loaded into the corresponding positions in the main attribute encoding area. During loading, starting from the first bit of the main attribute encoding area, the numeric identifier for the first attribute category is placed within the placeholder length starting from its starting position, according to the order of the attribute categories in the global enumeration dictionary. Then, the numeric identifier for the second attribute category is placed within the placeholder length starting from its starting position, and so on, until all attribute category numeric identifiers have been filled. For employees with no actual attribute value under a certain attribute category, no search operation is performed; instead, a pre-defined null code is directly filled into the placeholder length starting from the starting position of that attribute category in the main attribute encoding area. The null code is a special binary sequence with the same length as the valid numeric identifier and that does not conflict with any valid numeric identifier; for example, an all-1s pattern can be used to represent a null value. After the above filling is completed, a compact attribute feature code for the employee is generated.

[0025] Example 2: In specific implementation, please refer to Figure 3 A mechanism for monitoring changes to employee attribute information is implemented within the core human resource management system. When the attribute information of any employee changes within the core human resource management system, the change event triggers a recalculation process for the employee's attribute feature code. The recalculation process includes: retrieving the employee's complete actual attribute values ​​for each attribute category and the hierarchical attribute values ​​for each hierarchical dimension from the core human resource management system after the change; searching for the corresponding numerical identifier for each actual attribute value based on a pre-established global enumeration dictionary; and regenerating an updated attribute feature code based on the coding scheme version number, hierarchical identifier generation rules, attribute category order, and placeholder length.

[0026] After generating the updated attribute signature, the employee's employee ID and the updated attribute signature are assembled into a data synchronization message. The message body of the data synchronization message contains an employee ID field and an attribute signature field. The employee ID field stores the employee's unique identifier, and the attribute signature field stores the complete binary data of the updated attribute signature.

[0027] After assembly, the data synchronization message is broadcast to all business application servers via an enterprise-level message bus. The enterprise-level message bus adopts a publish-subscribe model, with the human resource management core system acting as the message producer, publishing the data synchronization message to a preset topic; all business application servers act as message consumers, subscribing to the same topic.

[0028] After receiving a data synchronization message, each business application server parses the employee ID and the updated attribute signature from the message. The business application server maintains a global employee signature mapping table in its local memory. This table stores the attribute signature data of all employees, using the employee ID as the key and the attribute signature as the value.

[0029] The business application server performs a lookup operation in the global employee feature code mapping table based on the parsed employee ID, locating the existing record corresponding to that employee ID. After locating the existing record, it replaces the attribute feature code in the existing record with the updated attribute feature code, completing the update of the employee's attribute feature code in local memory. The replacement operation uses an atomic write method, ensuring that the reading and comparison operations of other records in the global employee feature code mapping table are not affected during the update of a single record. After the update, the distributed employee feature code set on each business application server is consistent with the latest employee attribute information in the core human resource management system.

[0030] Example 3: In specific implementation, please refer to Figure 4 When the application server receives an attribute filtering request from a client, it parses the request and extracts all attribute categories used as filtering conditions, as well as the corresponding filtering attribute values ​​for each category. If the request specifies multiple filtering attribute values ​​for a single attribute category, the category and each value are extracted as separate filtering conditions.

[0031] Based on the starting position and placeholder length of each attribute category recorded in the global enumeration dictionary, a corresponding mask segment is assigned to each filter attribute category in the filter mask. Specifically, each attribute category in the global enumeration dictionary is traversed to determine its starting position and placeholder length in the filter mask. The starting position is the same as the starting position of the attribute category in the attribute feature code recorded in the global enumeration dictionary, and the placeholder length is the same as the placeholder length of the attribute category in the attribute feature code recorded in the global enumeration dictionary. Thus, a continuous mask segment is drawn in the filter mask that completely corresponds to the main attribute encoding area in the attribute feature code.

[0032] For the specified filter attribute category in the attribute filtering request, the numeric identifier corresponding to the filter attribute value under that filter attribute category is searched from the global enumeration dictionary. The found numeric identifier is then filled into the precise bits of the mask field allocated for that filter attribute category in the filter mask. When filling, the numeric identifier is written bit by bit, starting from the most significant or least significant bit of the mask field, in the same way as the numeric identifier is stored in the attribute feature code.

[0033] For attribute categories not specified in the attribute filtering request, no numeric identifier lookup operation is performed. Instead, all mask fields allocated for that unspecified attribute category in the filter mask are filled with wildcard bits. The wildcard bits in the filter mask are filled with a special binary value that is distinct from all valid numeric identifiers in the global enumeration dictionary. Optionally, the special binary value can be a binary string where every bit is 1, but this string is not used as a valid numeric identifier for any attribute category. The global enumeration dictionary is generated to ensure that the numeric identifier allocated to each attribute value does not use this special binary value.

[0034] For employee stratification filtering, when parsing attribute filtering requests, it checks whether the request contains filtering instructions for employee stratification. Employee stratification filtering instructions specify filtering values ​​for at least one of the following dimensions: personnel category, core department affiliation, and emergency contact status. When employee stratification filtering instructions are included, based on the arrangement order of the stratification identifier bits in the stratification identifier area and the placeholder length of each bit, a precise stratification value matching the filtering instruction is set at the corresponding position in the filtering mask. The precise stratification value is set by filling the binary encoded value corresponding to the dimension specified in the stratification filtering instruction into the corresponding stratification identifier bit position in the filtering mask; for stratification dimensions not specified in the stratification filtering instruction, a wildcard bit is filled into the stratification identifier bit of that dimension in the filtering mask. The final generated filtering mask has the same total length as the attribute feature code, including the header stratification identifier area mask segment and the subsequent main attribute encoding area mask segment. Thus, the incomplete attribute filtering conditions from the client are converted into a filtering mask of the same length as the attribute feature code.

[0035] Example 4: In practice, each business application server divides the distributed employee signature set stored in its local memory into multiple data segments. The process of segmenting the data includes: obtaining the total number of employee IDs in the distributed employee signature set; obtaining a predetermined number of scanning threads; calculating the number of employee signatures in each data segment based on the total number of employee IDs and the number of scanning threads; and dividing the consecutively arranged employee signature records into data segments equal to the number of scanning threads, according to the storage order of the employee IDs in the distributed employee signature set.

[0036] Each data segment is assigned an independent scanning thread, allowing multiple scanning threads to perform traversal tasks in parallel on their respective assigned data segments. Each scanning thread, upon creation, receives the starting and ending employee ID positions of the data segment assigned to it.

[0037] Each scanning thread, within its allocated data segment, sequentially reads the attribute feature code corresponding to each employee ID according to the storage order of the employee IDs. After reading an attribute feature code, the scanning thread extracts the hierarchical identifier segment from a fixed position within the attribute feature code. The starting position and length of the hierarchical identifier segment within the attribute feature code are determined by the structural definition of the attribute feature code; specifically, it is located after the header verification area and before the main attribute encoding area.

[0038] The scanning thread performs an initial comparison between the extracted hierarchical identifier segment and the corresponding hierarchical identifier segment in the filter mask. The starting position and length of the corresponding hierarchical identifier segment in the filter mask are exactly the same as the starting position and length of the hierarchical identifier segment in the attribute feature code. The initial comparison uses a bit-by-bit comparison method, comparing the value of each binary bit in the hierarchical identifier segment sequentially from the most significant bit to the least significant bit.

[0039] When the hierarchical identifier segment in the attribute feature code is not completely consistent with the corresponding hierarchical identifier segment in the filter mask, that is, at least one binary bit is different, the scanning thread determines that the current employee ID does not meet the filter conditions, skips the current employee ID, does not perform subsequent main attribute encoding area comparison operations, and continues to read the attribute feature code corresponding to the next employee ID in the data segment.

[0040] When the hierarchical identifier segment in the attribute feature code is completely identical to the corresponding hierarchical identifier segment in the filter mask (i.e., the value of each binary bit is the same), the scanning thread determines that the hierarchical identifier segment matches successfully and continues to perform a segment-by-segment bit-level comparison between the main attribute encoding area of ​​the attribute feature code and the corresponding mask bit segment in the filter mask. The segment-by-segment bit-level comparison process is as follows: the scanning thread extracts the first encoded sub-segment corresponding to the current attribute category from the attribute feature code according to the arrangement order of each attribute category in the main attribute encoding area. The starting position and placeholder length of the first encoded sub-segment in the attribute feature code are determined by the starting position and placeholder length of the attribute category recorded in the global enumeration dictionary. Simultaneously, the scanning thread extracts the second mask sub-segment corresponding to the current attribute category from the filter mask. The starting position and placeholder length of the second mask sub-segment in the filter mask are the same as the starting position and placeholder length of the first encoded sub-segment in the attribute feature code.

[0041] After extracting the first encoded segment and the second mask segment, the scanning thread determines whether all bits of the second mask segment are wildcard bits. Wildcard bits are filled with special binary values ​​that are different from all valid numeric identifiers in the global enumeration dictionary. When all bits of the second mask segment are this special binary value, the scanning thread confirms that the current attribute category matches successfully, does not perform a bit-by-bit comparison between the first encoded segment and the second mask segment, and continues to extract and compare the first encoded segment and the second mask segment corresponding to the next attribute category.

[0042] When at least one bit in the second mask segment is not a wildcard, the scanning thread performs a complete consistency comparison between each bit of the first encoded segment and the corresponding bit in the second mask segment. If the values ​​of each corresponding bit in the first encoded segment and the second mask segment are the same, the scanning thread confirms that the current attribute category matches successfully and continues to extract the first encoded segment and the second mask segment corresponding to the next attribute category for comparison. If the values ​​of any corresponding bit in the first encoded segment and the second mask segment are different, the scanning thread confirms that the current attribute category matches unsuccessfully and immediately terminates all subsequent comparison operations for the current employee ID, determining that the current employee ID does not meet the filtering conditions, skipping the current employee ID, and continuing to read the attribute feature code corresponding to the next employee ID in the data segment.

[0043] The scanning thread continues to execute the above comparison process until it completes the segment-by-segment, bit-level comparison of all attribute categories corresponding to the current employee ID. If every attribute category is confirmed to match, the employee ID is determined to meet the filtering criteria, and the scanning thread collects the employee ID into a temporary result set. Subsequently, the scanning thread continues processing the next employee ID in the data segment until it has traversed all the attribute feature codes corresponding to all employee IDs in its assigned data segment.

[0044] Example 5: In specific implementation, please refer to Figure 5 Each application server obtains the number of cores in its local CPU by calling the operating system interface. The number of cores in a local CPU refers to the number of physical or logical cores in the CPU that can independently execute instructions. The application server uses the obtained number of cores in its local CPU as the initial number of scan threads; that is, the initial number of scan threads is set equal to the number of cores in the local CPU.

[0045] Each application server retrieves the total number of employee IDs from the distributed employee signature set. The total number of employee IDs in the distributed employee signature set refers to the number of all distinct employee IDs contained in the employee signature mapping table stored in the local memory of the current application server. The application server iterates through the key set in the global employee signature mapping table to obtain the total number of employee IDs.

[0046] The application server combines the initial number of scanning threads and the total number of employee IDs to calculate the number of employee signatures in each data segment. The calculation method is as follows: divide the total number of employee IDs by the initial number of scanning threads, round the result up, and the integer value is the number of employee signatures in each data segment. When the total number of employee IDs is not divisible by the initial number of scanning threads, the last data segment contains fewer employee signatures than the other data segments.

[0047] Data segments are divided sequentially according to the storage order of employee IDs in the distributed employee signature set. The storage order of employee signatures in the distributed employee signature set refers to the arrangement order of employee IDs in the global employee signature mapping table, such as lexicographical order or insertion order. Starting with the first employee ID, consecutive employee signature records with the same number of signatures as the calculated data segments are extracted to form the first data segment; continuing from the next employee ID, the same number of consecutive employee signature records are extracted to form the second data segment; and so on, until all employee signature records have been divided. Each data segment contains a continuous sequence of employee signature records and can be uniquely identified by the positions of the first and last employee IDs within the segment.

[0048] Each application server allocates an independent scanning thread for each data segment. The allocation method is as follows: create the same number of scanning threads as the number of data segments, and bind each data segment to one scanning thread. The application server passes the starting employee ID position and ending employee ID position of each data segment to the scanning thread bound to that data segment. The starting employee ID position refers to the index or key value of the first employee ID in the data segment in the global employee signature mapping table; the ending employee ID position refers to the index or key value of the last employee ID in the data segment in the global employee signature mapping table.

[0049] After receiving the starting employee ID position and the ending employee ID position, each scanning thread independently begins to execute the traversal and comparison task of its bound data segment. The specific execution flow of the traversal and comparison task is consistent with the segmented comparison and bit-level matching operation of attribute feature codes and filter masks mentioned above in this embodiment. Each scanning thread starts from the starting employee ID position, sequentially reads the attribute feature code corresponding to each employee ID, performs the filtering and comparison operation, until the employee feature code record corresponding to the ending employee ID position is processed.

[0050] After each scanning thread completes the traversal and comparison of its initially bound data segment, it checks if any other data segments have not yet been traversed. The check is performed as follows: the application server maintains a shared task status tracking structure, which records the processing status of each data segment, including "unprocessed" and "completed." The scanning thread accesses this shared task status tracking structure to check for any data segments with an "unprocessed" status. When an unprocessed data segment is found, the scanning thread retrieves one of these segments, obtains its starting and ending employee ID positions, and continues traversing and comparing the newly retrieved data segments. The retrieval operation is atomic to prevent multiple scanning threads from retrieving the same data segment simultaneously. The scanning thread repeats the retrieval and data segment processing operations until the processing status of all data segments becomes "completed."

[0051] Once all data segments have been traversed—meaning every employee ID record in each segment has been compared by the scanning thread—each scanning thread aggregates the employee IDs that meet the filtering criteria into a thread-safe result set. The application server then summarizes all employee IDs in this thread-safe result set to form a complete real-time filtering result.

[0052] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A distributed real-time employee attribute filtering method based on compact coding, characterized in that, The method includes: A global enumeration dictionary is established for all filterable employee attributes in the human resource management system, and a variable-length numeric identifier is assigned to each attribute value. Based on the digital identifier, all attribute values ​​of each employee are encoded into a compact attribute feature code containing a header verification area, a hierarchical identifier area, and a main attribute encoding area, and the attribute feature code is used as a component of the employee master data; Each employee ID is paired with its corresponding attribute feature code to form a key-value pair, which is then distributed to the local memory of all business application servers through a data synchronization mechanism, thus forming a distributed set of employee feature codes. When an attribute filtering request is received from a client, the filtering conditions contained in the attribute filtering request are parsed, and the filtering conditions are converted into a filtering mask of the same length as the attribute feature code according to the global enumeration dictionary. Each business application server traverses the distributed employee feature code set in parallel in its local memory, performs segmented comparison and bit-level matching operations on the attribute feature code corresponding to each employee ID with the filtering mask, and filters out all employee IDs whose attribute feature codes completely match the filtering mask as the real-time filtering result.

2. The distributed real-time employee attribute filtering method based on compact coding according to claim 1, characterized in that, The process involves establishing a global enumeration dictionary for all filterable employee attributes in the human resource management system, and assigning a variable-length numeric identifier to each attribute value, including: All filterable employee attributes in the human resource management system are divided into attribute categories, and a category identifier field is assigned to each attribute category; For each specific attribute value under the same attribute category, the length of the numerical identifier of each attribute value is determined based on the estimated number of all possible values ​​within that attribute category, so that the numerical identifier of each attribute value is unique within the attribute category. The global enumeration dictionary is generated by summarizing the category identifier fields of all attribute categories and the numeric identifiers corresponding to each attribute value, and the starting position and placeholder length of each attribute category in the attribute feature code are recorded.

3. The distributed real-time employee attribute filtering method based on compact coding according to claim 1, characterized in that, Based on the digital identifier, all attribute values ​​of each employee are encoded into a compact attribute feature code containing a header verification area, a hierarchical identifier area, and a main attribute encoding area, including: Fill the fixed number of bits in the header verification area with the overall verification information of the encoding scheme version number and the attribute feature code; Based on the preset employee stratification dimensions, obtain the stratification attribute value of each employee under the stratification dimension, and convert the stratification attribute value into the stratification identifier bit in the corresponding position in the stratification identifier area. The stratification dimensions include personnel category dimension, core department affiliation dimension and emergency contact status dimension. Based on the actual attribute values ​​of each employee under each attribute category, the numerical identifier corresponding to each actual attribute value is obtained from the global enumeration dictionary. According to the starting position and place length of each attribute category recorded in the attribute feature code, the numerical identifiers are sequentially concatenated to the main attribute encoding area. For employees who have no actual attribute values ​​under a certain attribute category, a preset empty value code is filled in the corresponding position in the main attribute encoding area.

4. The distributed real-time employee attribute filtering method based on compact coding according to claim 1, characterized in that, The formation of the distributed employee signature set includes: When employee attribute information changes in the core human resource management system, the attribute feature code of the employee corresponding to the employee attribute information is recalculated to generate an updated attribute feature code. The employee ID and the updated attribute feature code are assembled into a data synchronization message, which is then broadcast to all business application servers via an enterprise-level message bus. After receiving the data synchronization message, each business application server searches for and replaces the attribute feature code corresponding to the employee ID in the global employee feature code mapping table maintained in local memory, based on the employee ID in the data synchronization message, thus completing the real-time update of the distributed employee feature code set.

5. The distributed real-time employee attribute filtering method based on compact coding according to claim 1, characterized in that, When an attribute filtering request is received from a client, the process of parsing the filtering conditions contained in the attribute filtering request and converting the filtering conditions into a filtering mask of the same length as the attribute feature code based on the global enumeration dictionary includes: Parse the attribute filtering request and extract all filtering attribute categories involved in the attribute filtering request and the filtering attribute value corresponding to each filtering attribute category; Based on the starting position and placeholder length of each attribute category recorded in the global enumeration dictionary, a corresponding mask bit segment is assigned to each filter attribute category in the filter mask; For the filter attribute category specified in the attribute filtering request, fill the numeric identifier corresponding to the filter attribute value into the precise bits of the mask field in the filter mask; For attribute categories not specified in the attribute filtering request, wildcard bits are filled into the mask field of the filtering mask; When the attribute filtering request contains a filtering instruction for employee stratification, the precise stratification value matching the filtering instruction is set at the corresponding position in the filtering mask according to the arrangement order of each stratification identifier bit in the stratification identifier area.

6. The distributed real-time employee attribute filtering method based on compact coding according to claim 1, characterized in that, Each business application server traverses the distributed employee feature code set in parallel in its local memory, performing segmented comparison and bit-level matching operations on the attribute feature code corresponding to each employee ID with the filter mask, and filtering out all employee IDs whose attribute feature codes completely match the filter mask, as the real-time filtering result, including: Each business application server divides the distributed employee signature set in local memory into multiple data segments and allocates an independent scanning thread to each data segment, so that multiple scanning threads can perform traversal tasks in parallel. Each scanning thread sequentially reads the attribute feature code corresponding to each employee ID from its assigned data segment, extracts the hierarchical identifier segment from the attribute feature code, and performs an initial comparison with the corresponding hierarchical identifier segment in the filter mask. When the hierarchical identifier segment in the attribute feature code is not completely consistent with the corresponding hierarchical identifier segment in the filter mask, the scanning thread skips the current employee ID and continues to process the next employee ID; When the hierarchical identifier segment in the attribute feature code is completely consistent with the corresponding hierarchical identifier segment in the filter mask, the scanning thread continues to perform segment-by-segment bit-level comparison between the main attribute encoding area of ​​the attribute feature code and the corresponding mask bit segment in the filter mask until the comparison of all bit segments is completed or a mismatched bit segment is encountered, at which point the comparison stops.

7. The distributed real-time employee attribute filtering method based on compact coding according to claim 6, characterized in that, The scanning thread continues to perform a segment-by-segment bit-level comparison between the main attribute encoding area of ​​the attribute feature code and the corresponding mask bit segment in the filtering mask, including: The scanning thread extracts the first encoded segment corresponding to the current attribute category from the attribute feature code in the order of the attribute categories in the main attribute encoding area, and extracts the second mask segment corresponding to the current attribute category from the filter mask. Determine whether each bit of the second mask sub-segment is a wildcard bit; When the second mask segment is the wildcard bit, the scanning thread confirms that the current attribute category matches successfully, and continues to extract the first encoded segment and the second mask segment corresponding to the next attribute category for comparison; When the second mask segment is not the wildcard bit, the scanning thread performs a complete consistency comparison between each bit of the first encoded segment and each bit of the second mask segment. If each bit is the same, the current attribute category is confirmed to be matched successfully. If any bit is different, the current attribute category is confirmed to be matched unsuccessfully, and the comparison operation for the current employee ID is terminated.

8. The distributed real-time employee attribute filtering method based on compact coding according to claim 6, characterized in that, Each business application server divides the distributed employee signature set in its local memory into multiple data segments and allocates an independent scanning thread to each data segment, including: Each business application server obtains the number of cores in its local central processing unit and determines the initial number of scan threads based on the number of cores. Each business application server obtains the total number of employee IDs in the distributed employee feature code set, calculates the number of employee feature codes in each data segment based on the initial number of scanning threads, and divides the data segments sequentially according to the storage order of employee IDs in the distributed employee feature code set. Each business application server allocates an independent scanning thread for each data segment and passes the start and end employee ID positions of each data segment to the corresponding scanning thread. After each scanning thread completes its assigned data segment traversal task, it retrieves a new data segment from other data segments that have not yet been traversed and continues execution until all data segments have been traversed.

9. The distributed real-time employee attribute filtering method based on compact coding according to claim 5, characterized in that, The wildcard bit is filled in the filter mask with a special binary value that is different from all the valid numeric identifiers.

10. A distributed real-time employee attribute screening system based on compact coding, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the distributed real-time employee attribute filtering method based on compact coding as described in any one of claims 1 to 9.

Images