A mac address intelligent aggregation method and system for home network device profiling

Abstract

The application provides a MAC address intelligent aggregation method and system for household network equipment portraits, which comprises the following steps: obtaining a MAC address associated with a terminal equipment model identifier; dividing the MAC address into multiple data fragments according to a manufacturer prefix and sorting to obtain an ordered MAC address sequence; performing multi-constraint greedy clustering on each fragment to define an extended window by a neighboring distance constraint, a model purity constraint and a density constraint, and determining a termination point according to interval density and interval coverage range to obtain a preliminary aggregation cluster; performing quality evaluation and filtering on the preliminary aggregation cluster by using a machine learning classification model; merging the aggregation clusters of the same model and adjacent addresses when the density meets the condition to form a final aggregation cluster table; and constructing an ordered index based on the start MAC address of the aggregation cluster to perform interval matching on the MAC address to be inferred to infer the terminal equipment model identifier. The application can improve the MAC address aggregation accuracy and processing efficiency.

Description

Technical Field

[0001] This invention relates to the fields of computer network security and large-scale data processing technology, and in particular to a method and system for intelligent aggregation of MAC addresses for profiling home network devices. Background Technology

[0002] With the rapid increase in the number of terminal devices in home networks, operators and network management systems need to accurately identify the models and build device profiles for these devices. Currently, the mainstream solution relies on matching fingerprint features reported by terminal devices to determine their model, manufacturer, type, and other profile attributes. However, given the ever-growing scale of terminal devices, relying solely on fingerprint matching faces challenges in terms of both recognition coverage and processing efficiency.

[0003] In existing technologies, to improve the identification coverage of terminal devices, a method has been proposed to construct continuous MAC address intervals based on identified MAC addresses, and then infer the model of devices that have not reported fingerprints. Using a single density threshold or a fixed adjacent spacing threshold as conditions, identified MAC addresses are scanned and expanded, and MAC addresses that meet the conditions are included in the same aggregate interval, ultimately generating a table mapping MAC address intervals to device models. After collecting the MAC addresses of devices that have not reported fingerprints, the device model can be inferred by finding the interval to which the MAC address belongs.

[0004] However, the above methods have the following problems in practical applications: First, when using a single threshold for interval expansion, a threshold that is too high can easily merge MAC addresses of different device models into the same interval, introducing cross-model contamination; a threshold that is too low will cause continuous MAC addresses of the same device model to be split into a large number of fragmented small intervals, losing continuity. Second, the quality of the aggregated intervals generated by clustering is inconsistent, lacking automated evaluation methods, relying on manual sampling and review, which is inefficient and difficult to update at scale. Third, when faced with massive MAC address data, the linear scanning processing method is time-consuming and consumes high computational resources, which cannot meet the needs of large-scale processing of billions of terminal data. Summary of the Invention

[0005] In view of the above problems, a method and system for intelligent MAC address aggregation for profiling home network devices is proposed to overcome or at least partially solve these problems. Specifically: A method for intelligent aggregation of MAC addresses for profiling home network devices includes: Obtain the MAC address associated with the terminal device model identifier; The MAC address is divided into several data segments according to the manufacturer prefix, and the MAC addresses in each data segment are sorted by integer value to obtain an ordered MAC address sequence. A parallel processing approach is adopted to perform multi-constraint greedy clustering on ordered MAC address sequences to obtain preliminary clusters. The multi-constraint greedy clustering defines the expansion window based on adjacent distance constraints, model purity constraints, and density constraints, and determines the termination point based on the interval density and interval coverage to form clusters. A machine learning classification model is used to evaluate the quality of the initial clusters, and clusters that meet the preset quality conditions are retained. For clusters of terminal devices with the same model identifier and adjacent MAC address spaces, they are merged when the density after merging meets the preset conditions to form the final cluster table. An ordered index is constructed based on the starting MAC address of the cluster in the final cluster table. Range matching is performed on the MAC address to be predicted that is not associated with the terminal device model identifier, and the terminal device model identifier corresponding to the MAC address to be predicted is inferred.

[0006] Optionally, the MAC addresses are divided into multiple data fragments based on the manufacturer prefix, and the MAC addresses within each data fragment are sorted by integer value, including: The MAC addresses are standardized and preprocessed by converting them into a uniform 12-digit hexadecimal string with no uppercase separators, and removing MAC address records containing illegal characters or invalid formats. The standardized preprocessed MAC address is resolved into a 48-bit unsigned integer. Extract the high 24 bits of the 48-bit unsigned integer as the manufacturer prefix; Based on the vendor prefix, MAC addresses with the same vendor prefix are grouped into the same data segment, and different vendor prefixes correspond to different data segments. Each data segment carries a non-overlapping MAC address subspace. For each data segment, the MAC addresses are sorted in ascending order according to their corresponding 48-bit unsigned integer values, and the sorted MAC addresses are deduplicated to obtain an ordered MAC address sequence.

[0007] Optionally, a parallel processing approach is adopted to perform multi-constraint greedy clustering on the ordered MAC address sequence to obtain preliminary clusters, including: Each data shard is assigned an independent processing process, and multiple processing processes execute in parallel, with each processing process being independent of the others; Each processing process independently performs multi-constraint greedy clustering on the ordered MAC address sequence corresponding to its data shard, generating a set of aggregated clusters corresponding to its data shard. After all processing processes are completed, the aggregated clusters corresponding to each data shard are merged. The clusters are deduplicated using the triplet of the starting MAC address, ending MAC address, and terminal device model identifier as the deduplication key, and then sorted in ascending order by the starting MAC address to form a preliminary aggregated cluster.

[0008] Optionally, in each processing step, multi-constraint greedy clustering generates each cluster in the set of clusters corresponding to the data shard by repeatedly executing the following steps: Starting from the MAC address at the current position in the ordered MAC address sequence, record the terminal device model identifier corresponding to the starting point, and record the number of MAC addresses that have been included in the current cluster and the number of different MAC addresses that have been skipped during the scan. Starting from the starting point, scan the candidate MAC addresses one by one in the direction of increasing integer values ​​in the ordered MAC address sequence; During the scanning process, when the integer difference between a candidate MAC and the previous MAC exceeds the preset maximum adjacent distance, the adjacent distance constraint is terminated; when the terminal device model identifier of the candidate MAC is different from the terminal device model identifier corresponding to the starting point, the number of skipped different model MACs is increased by one; when the increased number of skipped different model MACs exceeds the product of the number of MACs already included in the current cluster and the preset maximum skip ratio, the model purity constraint is terminated; when the density calculated based on the ratio of the number of MACs already included in the current cluster minus one to the integer difference between the candidate MAC and the starting point is lower than the preset minimum density threshold, the density constraint is terminated. The scan stops when any constraint is triggered. Among the candidate MAC addresses scanned before the scan stops, the endpoint MAC address is selected based on the interval density and interval coverage, and an aggregated cluster containing the starting point, the endpoint MAC address, and the terminal device model identifier corresponding to the starting point is output.

[0009] Optionally, the endpoint MAC address can be selected based on the density and coverage of the interval, including: Among the candidate MACs scanned before the scan stops, each candidate MAC is used as a candidate termination point; For each candidate termination point, a comprehensive score is calculated based on the density of the interval and the coverage of the interval corresponding to the candidate termination point; The candidate termination point with the highest overall score is selected as the endpoint MAC address; The overall score is calculated as follows: score=α×density+β×log(distance+1) Where α is the density weight, β is the span weight, density is the interval density corresponding to the candidate termination point, and distance is the integer difference between the candidate termination point and the starting point.

[0010] Optionally, a machine learning classification model is used to evaluate the quality of the initial clusters, retaining clusters that meet preset quality conditions, including: For each cluster in the initial cluster, a five-dimensional feature vector is extracted. The five-dimensional feature vector includes: the terminal device model identifier corresponding to the cluster, the number of aggregated MAC addresses in the cluster, the cluster density, the cluster span, and the cluster's overall score. The density is the ratio of the number of aggregated MAC addresses minus one to the integer difference between the cluster's endpoint MAC address and its starting point MAC address. The span is the integer difference between the cluster's endpoint MAC address and its starting point MAC address plus one. The five-dimensional feature vector is input into the machine learning classification model, and the machine learning classification model outputs a quality prediction label. The quality prediction label is used to indicate whether the cluster belongs to a high-quality cluster or a low-quality cluster. Clusters whose quality prediction labels indicate high quality are retained, while clusters whose quality prediction labels indicate low quality are removed.

[0011] Optionally, the machine learning classification model is a random forest model, and the training process of the random forest model includes: Several cluster records are sampled from the historical aggregation results. A five-dimensional feature vector is extracted from each cluster record, and quality labels are labeled to form a training sample set. The quality labels include high-quality labels and low-quality labels. Using a five-dimensional feature vector as input and quality labels as output, random search cross-validation is used to search for the hyperparameters of the random forest model. The hyperparameters include the number of decision trees, maximum depth, and minimum number of sample splits. During cross-validation, hyperparameter combinations that meet preset conditions for classification performance metrics on the validation set are selected to obtain the trained random forest model, and the trained random forest model is persistently saved.

[0012] Optionally, clusters with the same terminal device model identifier and adjacent MAC address spaces are merged when the density after merging meets preset conditions, including: The remaining clusters are arranged in ascending order of their starting MAC addresses to form a sequence of clusters to be merged. Traverse the sequence of clusters to be merged. For two adjacent clusters with the same terminal device model identifier, calculate the new interval density after merging the two adjacent clusters. When the density of the new interval after merging is not lower than the preset merging density threshold, two adjacent clusters are merged into one cluster. The merged cluster is identified by the starting MAC address of the first cluster of the two adjacent clusters, the ending MAC address of the second cluster of the two adjacent clusters, and the terminal device model identifier. When the density of the new merged interval is lower than the preset merge density threshold, two adjacent clusters are retained as independent clusters. The calculation of the new interval density after merging two adjacent clusters includes: Obtain the number of aggregated MAC addresses in the previous cluster and the number of aggregated MAC addresses in the next cluster, and calculate the total number of aggregated MAC addresses after merging, which is the sum of the two aggregated MAC address counts. Obtain the integer difference between the endpoint MAC address of the next aggregation cluster and the starting MAC address of the previous aggregation cluster, and calculate the number of MAC addresses covered by the new interval after merging as the integer difference plus one; The ratio of the total number of aggregated MAC addresses to the number of MAC addresses covered by the new merged interval is calculated and used as the density of the new merged interval.

[0013] Optionally, an ordered index is constructed based on the starting MAC address of the cluster in the final cluster table. Range matching is then performed on the MAC addresses to be predicted that are not associated with any terminal device model identifier to predict the terminal device model identifier corresponding to the MAC address, including: Sort the final aggregated cluster table in ascending order by the starting MAC address; Extract the starting MAC address of each cluster from the final cluster table after sorting to form an ordered array of starting MAC addresses; The endpoint MAC address and terminal device model identifier of each cluster are arranged in the same order as the ordered starting MAC address array to form the endpoint MAC address array and the terminal device model identifier array, respectively. Convert the MAC address to be inferred into an integer value; Perform a binary search on the ordered array of starting MAC addresses to locate the target index, which is the maximum index value that satisfies the condition that the starting MAC address does not exceed the MAC address to be predicted. Read the endpoint MAC address and terminal device model identifier corresponding to the target index; When the MAC address to be predicted is less than or equal to the destination MAC address, the terminal device model identifier of the MAC address to be predicted is predicted as the terminal device model identifier corresponding to the target index. When the MAC address to be inferred is greater than the endpoint MAC address, the MAC address to be inferred is marked as unmatched.

[0014] A MAC address intelligent aggregation system for profiling home network devices includes: The data fragmentation preprocessing module is used to obtain the MAC address associated with the terminal device model identifier, divide the MAC address into multiple data fragments according to the manufacturer prefix, sort the MAC addresses in each data fragment by integer value, and obtain an ordered MAC address sequence. The parallel clustering module is used to perform multi-constraint greedy clustering on ordered MAC address sequences in a parallel processing manner to obtain preliminary clusters. The multi-constraint greedy clustering defines the expansion window based on adjacent distance constraints, model purity constraints, and density constraints, and determines the termination point based on the interval density and interval coverage to form clusters. The intelligent quality assessment module is used to perform quality assessment on the initial clusters using a machine learning classification model, and retain the clusters that meet the preset quality conditions. The interval merging module is used to merge clusters of terminal devices with the same model identifier and adjacent MAC address spaces when the density after merging meets the preset conditions, to form the final cluster table. The inference matching module is used to construct an ordered index based on the starting MAC address of the aggregated cluster in the final aggregated cluster table, perform range matching on the MAC address to be inferred that is not associated with the terminal device model identifier, and infer the terminal device model identifier corresponding to the MAC address to be inferred.

[0015] This invention employs a multi-constraint greedy clustering mechanism, simultaneously applying adjacent spacing constraints, model purity constraints, and density constraints during the aggregation process. These three constraints jointly define the expansion window, and the termination point is determined within the expansion window based on the density and coverage of the intervals. In principle, this avoids the problems of heterogeneous model contamination and interval fragmentation caused by improper threshold settings in a single threshold scheme. This makes the MAC address intervals generated by aggregation more closely match the actual address allocation rules of terminal equipment manufacturers, thereby improving the accuracy of model inference for unreported fingerprint terminal equipment. This invention also introduces a machine learning classification model to automatically assess the quality of the initial aggregated clusters, and objectively screens the aggregation results based on the multidimensional features of the aggregated clusters, replacing the manual sampling and review process. This standardizes and makes the quality assessment process of the feature library reproducible, supports high-frequency iterative updates of the feature library, and reduces human maintenance costs. This invention also segments the MAC address space by manufacturer prefix, leveraging the inherent contiguous nature of MAC addresses from the same manufacturer to divide global data into independent subspaces. Combined with parallel processing, clustering tasks are performed synchronously on each segment, reducing the processing time for massive MAC address data as the parallel scale expands, thus meeting the large-scale processing needs of billions of terminal data points. Simultaneously, an ordered index is constructed based on the starting MAC address of the final aggregated cluster table, and binary interval matching is performed on the MAC addresses to be inferred. The time complexity of a single inference is logarithmic, efficiently supporting batch model inference for hundreds of millions of terminal devices, thereby improving the identification coverage and completeness of home network device profiles. Attached Figure Description

[0016] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a method for intelligent aggregation of MAC addresses for profiling home network devices, provided by an embodiment of the present invention. Figure 2 This is a flowchart of the clustering algorithm provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the comprehensive scoring function provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the system modules provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] Reference Figures 1-4 This invention provides a method for intelligent aggregation of MAC addresses for profiling home network devices, which may specifically include: A method for intelligent aggregation of MAC addresses for profiling home network devices includes: Obtain the MAC address associated with the terminal device model identifier; The MAC address is divided into several data segments according to the manufacturer prefix, and the MAC addresses in each data segment are sorted by integer value to obtain an ordered MAC address sequence. A parallel processing approach is adopted to perform multi-constraint greedy clustering on ordered MAC address sequences to obtain preliminary clusters. The multi-constraint greedy clustering defines the expansion window based on adjacent distance constraints, model purity constraints, and density constraints, and determines the termination point based on the interval density and interval coverage to form clusters. A machine learning classification model is used to evaluate the quality of the initial clusters, and clusters that meet the preset quality conditions are retained. For clusters of terminal devices with the same model identifier and adjacent MAC address spaces, they are merged when the density after merging meets the preset conditions to form the final cluster table. An ordered index is constructed based on the starting MAC address of the cluster in the final cluster table. Range matching is performed on the MAC address to be predicted that is not associated with the terminal device model identifier, and the terminal device model identifier corresponding to the MAC address to be predicted is inferred.

[0020] The MAC address intelligent aggregation method proposed in this invention constructs a high-confidence MAC address interval by mining the distribution pattern of MAC addresses associated with terminal device model identifiers in the numerical space, providing a basis for model inference for MAC addresses not associated with terminal device model identifiers.

[0021] Specifically, the process begins by loading MAC addresses associated with terminal device model identifiers and active, yet-to-be-inferred MAC addresses without associated terminal device model identifiers from the raw big data storage system as input data. The raw big data storage system can be a ClickHouse columnar database, an HDFS distributed file system, or other big data storage platforms that support massive data storage and batch reading. MAC addresses associated with terminal device model identifiers are used for subsequent cluster generation, while active, yet-to-be-inferred MAC addresses without associated terminal device model identifiers will be used for model inference after the final cluster table is established. Each loaded identified record contains a complete MAC address and the corresponding terminal device model identifier. The terminal device model identifier uniquely distinguishes different models of devices from different manufacturers. A MAC address, or Media Access Control address, is a unique hardware identifier assigned by the manufacturer to a network device at the time of manufacture, consisting of 48 binary bits.

[0022] After acquiring the input data, the MAC address is divided into multiple data fragments based on the vendor prefix. The vendor prefix, which is the high 24 bits of the MAC address, identifies the network device's manufacturer. When a manufacturer obtains a specific vendor prefix, it assigns MAC addresses to its network devices sequentially within the address space defined by that prefix. MAC addresses of the same model of device often cluster within consecutive address segments under the same vendor prefix. Based on this allocation characteristic, MAC addresses with the same vendor prefix are grouped into the same data fragment, with different vendor prefixes corresponding to different data fragments. Each data fragment carries a non-overlapping MAC address subspace. There are no data dependencies between the data fragments, providing naturally independent task units for parallel processing.

[0023] After data sharding is completed, the MAC addresses within each data shard are sorted by integer value and deduplicated, resulting in an ordered sequence of MAC addresses for each data shard. The sorting operation reorganizes the MAC address records within a shard according to their numerical spatial proximity, ensuring that addresses that are close in numerical space are also adjacent in the sequence. The deduplication process removes duplicate records caused by repeated reporting from different times or different data sources, preventing duplicate counting from interfering with subsequent density calculations.

[0024] After obtaining the ordered MAC address sequence of each data shard, a parallel processing approach is adopted, distributing the clustering tasks of each shard to different processing processes for synchronous execution. The processing processes are independent of each other, with no shared state and no need for inter-process communication coordination. The number of processing processes can be configured based on the number of available processor cores and the total number of data shards. When the number of data shards exceeds the number of available processing processes, a process pool mechanism can be used for task scheduling.

[0025] Within each processing step, a multi-constraint greedy clustering algorithm is performed on the ordered MAC address sequence of its respective segment. The clustering process starts with the currently unprocessed MAC record in the sequence, uses the terminal device model identifier corresponding to the starting point as the model attribution criterion, and scans candidate MACs one by one in the increasing direction of the integer value of the sequence. During the scanning process, adjacent spacing constraints, model purity constraints, and density constraints are applied simultaneously, and the three constraints together define an expandable legal window.

[0026] Adjacent spacing constraints identify significant breaks in the address space caused by different batches or cross-model allocations by determining whether the integer difference between a candidate MAC and the previous MAC exceeds a preset maximum adjacent spacing. Model purity constraints record the number of skipped MACs of different models during the scan; expansion terminates when the proportion of different models exceeds the product of the number of included MACs and a preset maximum skip ratio, preventing the erroneous merging of address segments from different models. Density constraints calculate the current density by dividing the number of included MACs by one by the integer difference between the candidate MAC and the starting point; expansion terminates when the density falls below a preset minimum density threshold, preventing cluster aggregations from crossing large address gaps lacking identified MAC support.

[0027] The scanning stops when any of the three constraints is triggered. Among the candidate MAC addresses that passed the constraint checks before stopping, the termination point is determined based on interval density and coverage. This is achieved by calculating a comprehensive score for each candidate MAC address within an expanded window, selecting the candidate MAC address with the highest comprehensive score as the termination point. This ensures that the generated clusters have sufficient density to support their confidence while extending coverage as much as possible to encompass more unidentified MAC addresses. After determining the termination point, the cluster record is output, containing the starting MAC address, the termination MAC address, and the terminal device model identifier corresponding to the starting point. The scan jumps from the starting point to the termination point, and the next round of clustering continues until the entire ordered MAC address sequence has been traversed.

[0028] After each shard is clustered, the aggregated cluster records output by all shards are collected and summarized. The records are merged and deduplicated using the triplet of the starting MAC address, the ending MAC address, and the terminal device model identifier as the deduplication key. The records are then sorted in ascending order by the starting MAC address to form a preliminary aggregated cluster set.

[0029] After obtaining the initial set of aggregated clusters, a machine learning classification model is used to evaluate the quality of the clusters in the set. A multi-dimensional feature vector is extracted for each cluster, containing the terminal device model identifier, the number of aggregated MAC addresses, density, span, and a comprehensive score. The feature vector is input into a pre-trained machine learning classification model, which outputs a quality prediction label indicating whether the cluster is high-quality or low-quality. Clusters predicted as high-quality are retained, while those predicted as low-quality are removed. The training process of the machine learning classification model can be completed offline. A training sample set is formed by sampling historical aggregation results and performing manual or semi-automatic annotation. The model parameters are then learned and optimized using this training sample set.

[0030] For the clusters retained after quality assessment, an adjacent interval merging operation is performed. The retained clusters are sorted in ascending order by their starting MAC address. For two adjacent clusters with the same terminal device model identifier, the density of the merged new interval is calculated. The density of the merged new interval is the sum of the aggregated MAC addresses of the two clusters divided by the number of MAC addresses covered by the merged new interval, where the number of MAC addresses covered by the merged new interval is the integer difference between the endpoint MAC address of the latter cluster and the starting MAC address of the former cluster plus one. When the density of the merged new interval is not lower than a preset merging density threshold, the two clusters are merged into one. The merged cluster uses the starting MAC address of the former cluster as its starting point, the ending MAC address of the latter cluster as its ending point, and the common terminal device model identifier as its model identifier. When the density of the merged new interval is lower than the preset merging density threshold, the two clusters are retained as independent clusters. After traversal, the final cluster table is obtained.

[0031] After the final cluster table is established, an ordered index is constructed based on the starting MAC addresses of each cluster. The final cluster table is sorted in ascending order by starting MAC addresses, and the starting MAC addresses of each cluster are extracted to form an ordered array of starting MAC addresses. Simultaneously, the ending MAC addresses and terminal device model identifiers of each cluster are stored in the ending MAC address array and the terminal device model identifier array, respectively, in the same order. For MAC addresses to be predicted that are not associated with a terminal device model identifier, they are converted to integer values, and a binary search is performed on the ordered array of starting MAC addresses to locate the largest index that satisfies the condition that the starting MAC address does not exceed the MAC address to be predicted. The ending MAC address and terminal device model identifier corresponding to this index are then read. If the MAC address to be predicted is less than or equal to the ending MAC address, its terminal device model identifier is predicted as the terminal device model identifier corresponding to that cluster; if it is greater than the ending MAC address, it is marked as a mismatch. The prediction results can be used to complete the model information of unreported fingerprint devices in the home network, improving the completeness of the device profile.

[0032] The preset parameters involved in the embodiments of the present invention, including the preset maximum adjacent spacing, the preset maximum skip ratio, the preset minimum density threshold, the density weight, the span weight, and the preset merging density threshold, can be configured and adjusted according to the manufacturer distribution characteristics and business needs of terminal devices in the network environment during actual deployment.

[0033] In one or more embodiments of the present invention, the MAC address is divided into multiple data fragments according to the manufacturer prefix, and the MAC addresses within each data fragment are sorted by integer value, including: The MAC addresses are standardized and preprocessed by converting them into a uniform 12-digit hexadecimal string with no uppercase separators, and removing MAC address records containing illegal characters or invalid formats. The standardized preprocessed MAC address is resolved into a 48-bit unsigned integer. Extract the high 24 bits of the 48-bit unsigned integer as the manufacturer prefix; Based on the vendor prefix, MAC addresses with the same vendor prefix are grouped into the same data segment, and different vendor prefixes correspond to different data segments. Each data segment carries a non-overlapping MAC address subspace. For each data segment, the MAC addresses are sorted in ascending order according to their corresponding 48-bit unsigned integer values, and the sorted MAC addresses are deduplicated to obtain an ordered MAC address sequence.

[0034] During the MAC address standardization preprocessing stage, the loaded MAC address records undergo format unification and validity verification. Because MAC address representation formats differ across data sources—some records are in the form of six groups of two-digit hexadecimal numbers separated by colons, while others are separated by hyphens or continuous without separators—and the case of uppercase and lowercase letters is inconsistent, a format unification operation needs to be performed on each MAC address string: all letter characters in the string are converted to uppercase, and colons, hyphens, periods, and other separators, as well as leading and trailing whitespace, are removed, retaining only continuous hexadecimal character sequences.

[0035] The sequence is then validated. A valid MAC address, after the above processing, should result in a string of length 12, where each character belongs to the hexadecimal character set, which consists of the digits 0-9 and the letters A-F. Records shorter than 12 characters or containing characters outside the hexadecimal character set are considered invalid and removed from the dataset. For multicast MAC addresses used for special purposes in the network, the least significant bit of the first byte is 1, which can be used to distinguish and filter them from unicast addresses during the preprocessing stage. After the above cleaning and validation, a standardized set of MAC address strings with a uniform format is obtained, each string being a 12-bit uppercase hexadecimal sequence without separators.

[0036] The standardized MAC address string is parsed into a 48-bit unsigned integer. The parsing process follows hexadecimal positional encoding rules: from left to right, the first bit of the standardized 12-bit string is the most significant bit, representing the highest 4 bits of the 48-bit integer; the 12th bit is the least significant bit, representing the lowest 4 bits of the 48-bit integer. Starting from the first bit, each hexadecimal character is mapped to a decimal value from 0 to 15, and this value is multiplied by its corresponding place value and then summed. The place value is base 16, with the exponent decreasing sequentially from the most significant bit: the first bit corresponds to 16 to the power of 11, the second bit to 16 to the power of 10, and so on, up to the 12th bit to 16 to the power of 0. The sum results in a value ranging from 0 to 2. 48 A 48-bit unsigned integer between -1. Using a 48-bit unsigned integer as the internal representation of the MAC address preserves all address space information of the MAC address, and the integer type directly supports subtraction operations to obtain the numerical range between addresses without the need for string comparison conversion.

[0037] After integer parsing, the high 24 bits are extracted from each 48-bit unsigned integer as a vendor prefix. This extraction is performed by right-shifting the 48-bit unsigned integer by 24 bits. The remaining high 24 bits are the organization-unique identifier for that MAC address, used to identify the network device's manufacturer. The allocation of this organization-unique identifier is globally unique; different manufacturers are assigned distinct organization-unique identifiers.

[0038] Data fragmentation is performed based on the extracted vendor prefixes. MAC addresses with the same vendor prefix value and their corresponding terminal device model identifiers are grouped into the same data fragment, while different vendor prefix values ​​correspond to different data fragments. Since the organization unique identifier has a fixed length of 24 bits, the global MAC address space can theoretically be divided into 2... 24 Each data fragment consists of a non-overlapping subspace, with each subspace comprising all possible MAC addresses sharing the same vendor prefix. In real-world network environments, the number of allocated and actively used vendor prefixes is far less than the theoretical upper limit; therefore, the actual number of data fragments generated depends on the number of vendor prefix types actually appearing in the input data. All MAC addresses contained within each data fragment share the same high 24 bits of the vendor prefix, differing only in the low 24 bits of the device identifier portion.

[0039] After data sharding, the MAC addresses within each shard are sorted in ascending order by their corresponding 48-bit unsigned integer values, and the sorted sequence is then deduplicated. The input data may contain instances where the same MAC address is reported repeatedly at different times or from different data sources. Without deduplication, this address would be counted repeatedly in the aggregated clusters, artificially inflating the interval density and affecting the accuracy of density constraints. Deduplication leverages the characteristic of consecutively adjacent identical MAC addresses in the sorted sequence. It iterates through the sorted sequence, comparing the MAC address integer values ​​of the current record with those of the previous record. If they are equal, only one record is retained. For conflicting cases where the same MAC address is associated with different terminal device model identifiers in different reported records, the record with higher credibility can be retained based on the credibility of the data source, or the MAC address can be removed from the dataset. After deduplication, each MAC address appears uniquely in the sequence, and the records are arranged in ascending order of their MAC address integer values, forming the ordered MAC address sequence corresponding to that data shard. The ordered MAC address sequence of each shard is carried in an independent data file and serves as input for the subsequent parallel clustering stage.

[0040] In one or more embodiments of the present invention, a parallel processing approach is used to perform multi-constraint greedy clustering on ordered MAC address sequences to obtain preliminary aggregated clusters, including: Each data shard is assigned an independent processing process, and multiple processing processes execute in parallel, with each processing process being independent of the others; Each processing process independently performs multi-constraint greedy clustering on the ordered MAC address sequence corresponding to its data shard, generating a set of aggregated clusters corresponding to its data shard. After all processing processes are completed, the aggregated clusters corresponding to each data shard are merged. The clusters are deduplicated using the triplet of the starting MAC address, ending MAC address, and terminal device model identifier as the deduplication key, and then sorted in ascending order by the starting MAC address to form a preliminary aggregated cluster.

[0041] This embodiment of the invention employs a parallel processing approach to perform multi-constraint greedy clustering on the ordered MAC address sequences of each data fragment obtained in the preceding steps, thereby obtaining preliminary aggregated clusters. This step leverages the non-overlapping nature of the MAC address spaces among the data fragments to decompose the global clustering task into multiple independent subtasks that are executed in parallel, thus shortening the processing time for massive amounts of MAC address data.

[0042] At the parallel architecture level, each data shard is assigned an independent processing process. The mapping between data shards and processing processes is one-to-one or many-to-one: when the number of available processing processes is greater than or equal to the number of data shards, each data shard is assigned to a dedicated processing process, and all processes start and execute synchronously; when the number of data shards exceeds the number of available processing processes, a process pool mechanism can be used for task scheduling. Each processing process automatically retrieves the next shard to be processed from the task queue after completing the clustering task for its current shard, until all shards have been processed. Each processing process runs independently, with no shared memory areas, no inter-process communication overhead, and no contention for access to shared data resources. Each processing process only reads the ordered MAC address sequence corresponding to the data shard assigned to it and performs clustering calculations in its private memory space, outputting a set of aggregated clusters belonging to that shard. This shared-nothing architecture allows the parallel efficiency of the clustering phase to increase approximately linearly with the number of processing processes. The upper limit of the number of processing processes depends on the number of available processor cores and can be flexibly set according to hardware configuration.

[0043] Within each processing step, multi-constraint greedy clustering is independently performed on the ordered MAC address sequence corresponding to the data shard. The clustering operation starts with the currently unprocessed MAC record in the sequence, using the terminal device model identifier corresponding to the starting point as the model attribution criterion, and scans candidate MACs one by one in ascending order of integer values ​​in the sequence. During the scan, adjacency spacing constraints, model purity constraints, and density constraints are applied simultaneously, collectively defining a scalable legal window. Adjacency spacing constraints identify significant breaks in the address space caused by different batches or cross-model allocations; model purity constraints control the proportion of MACs of different models mixed in within a cluster; and density constraints control the sparsity of the distribution of identified MACs within a cluster. Scanning stops when any of the three constraints is triggered. Among the candidate MACs that passed the constraint check before stopping, the termination point is determined based on a combination of interval density and interval coverage, outputting a cluster record containing the starting MAC address, the termination MAC address, and the terminal device model identifier corresponding to the starting point. Afterward, the scan jumps from the starting point to the termination point, and the next round of clustering continues. This process is repeated, generating one cluster record per round, until the scan position pointer reaches the end of the ordered MAC address sequence, at which point the clustering task for that shard is complete. Each cluster record generated in each round is stored in the aggregation result list of the processing process, forming the cluster set corresponding to the data shard.

[0044] After all processing processes complete their clustering tasks, the main process collects the aggregated cluster sets output by each processing process and performs cross-shard merging. Since the data shards do not overlap at the vendor prefix level, and the aggregated clusters of different shards are naturally separated in the address space, in most cases, it is not necessary to connect and merge aggregated clusters across shard boundaries. The main task of the merging operation is to summarize the clustering results of each shard and perform deduplication. Deduplication uses the triplet of the aggregated cluster's starting MAC address, ending MAC address, and terminal device model identifier as the deduplication key: if two aggregated cluster records have completely identical triplets, they are considered duplicate records, and only one is retained. After deduplication, all aggregated cluster records are sorted in ascending order by starting MAC address, forming a preliminary global aggregated cluster set. This preliminary aggregated cluster set will serve as input data for the subsequent machine learning quality assessment stage.

[0045] In one or more embodiments of the present invention, in each processing process, multi-constraint greedy clustering generates each cluster in the set of clusters corresponding to the data shard by repeatedly executing the following steps: Starting from the MAC address at the current position in the ordered MAC address sequence, record the terminal device model identifier corresponding to the starting point, and record the number of MAC addresses that have been included in the current cluster and the number of different MAC addresses that have been skipped during the scan. Starting from the starting point, scan the candidate MAC addresses one by one in the direction of increasing integer values ​​in the ordered MAC address sequence; During the scanning process, when the integer difference between a candidate MAC and the previous MAC exceeds the preset maximum adjacent distance, the adjacent distance constraint is terminated; when the terminal device model identifier of the candidate MAC is different from the terminal device model identifier corresponding to the starting point, the number of skipped different model MACs is increased by one; when the increased number of skipped different model MACs exceeds the product of the number of MACs already included in the current cluster and the preset maximum skip ratio, the model purity constraint is terminated; when the density calculated based on the ratio of the number of MACs already included in the current cluster minus one to the integer difference between the candidate MAC and the starting point is lower than the preset minimum density threshold, the density constraint is terminated. The scan stops when any constraint is triggered. Among the candidate MAC addresses scanned before the scan stops, the endpoint MAC address is selected based on the interval density and interval coverage, and an aggregated cluster containing the starting point, the endpoint MAC address, and the terminal device model identifier corresponding to the starting point is output.

[0046] Each processing process independently performs multi-constraint greedy clustering on the ordered MAC address sequence corresponding to its data shard. It generates each cluster in the cluster set corresponding to its data shard by repeatedly executing the following steps. The starting point of a single clustering process is the MAC record in the ordered MAC address sequence that has not yet been clustered. The position of this record is indicated by the scan position pointer maintained internally by the processing process.

[0047] At the start of clustering, the MAC record currently pointed to by the scan position pointer is used as the starting point. The integer value of the MAC address is extracted from this record as the starting MAC address, and the terminal device model identifier associated with this record is extracted as the model attribution basis for the current cluster. Two real-time state variables are created for the current cluster and assigned initial values: the number of MACs already included in the current cluster is initialized to 1, indicating that the starting MAC has been included; the number of different MACs skipped during the scan is initialized to 0, indicating that no different MACs have been skipped.

[0048] Starting from the starting point, proceed sequentially along the integer-increasing direction of the ordered MAC address sequence, reading each MAC record after the starting point as a candidate MAC. During the scanning of each candidate MAC, the adjacent spacing constraint, model purity constraint, and density constraint are judged in sequence, with the judgment order of the three constraints being consistent with the order listed above.

[0049] The adjacent spacing constraint is determined as follows: Calculate the difference between the integer value of the current candidate MAC address and the integer value of the previous MAC address. Here, the previous MAC address refers to the record immediately preceding the current candidate MAC address in the scan sequence; it is not necessarily the starting point, nor is it necessarily a MAC address already included in the aggregated cluster. If this integer difference exceeds the preset maximum adjacent spacing, it indicates that a gap exceeding the normal fluctuation range has appeared in the address space between the current candidate MAC address and the previous MAC address. In actual address allocation, manufacturers may experience occasional address jumps within consecutive address segments of the same model of equipment, such as individual address gaps caused by production line batch switching or reserved address intervals. However, the magnitude of such jumps is usually small. When the difference exceeds the preset threshold, this gap may mark the boundary of MAC address segments of different batches or different models of equipment, triggering the adjacent spacing constraint, and the scan terminates. The current candidate MAC address is not included in the aggregated cluster. If the difference does not exceed the preset threshold, the adjacent spacing constraint passes, and the model purity constraint judgment continues.

[0050] The method for determining model purity constraints is as follows: The terminal device model identifier associated with the current candidate MAC address is compared with the terminal device model identifier corresponding to the starting point. If they are the same, it means the candidate MAC address and the starting point belong to the same model device, the model purity constraint passes, and the density constraint determination continues. If they are different, it means the candidate MAC address belongs to a different model device, the number of skipped different model MAC addresses is increased by 1, and then it is determined whether the increased skipped number exceeds the product of the number of MAC addresses already included in the current cluster and the preset maximum skip ratio. The preset maximum skip ratio is a proportional parameter between 0 and 1, used to control the upper limit of the tolerable proportion of different model MAC addresses within the cluster. If the skipped number does not exceed this product, it means the model purity of the current cluster is still within an acceptable range, the model purity constraint passes, and although the different model MAC address is not counted in the already included number, it does not block expansion, and the density constraint determination continues. If the skipped number exceeds this product, it means the proportion of different model MAC addresses has exceeded the acceptable limit, and continued expansion will lead to the incorrect merging of address segments of different model devices, triggering the model purity constraint, and the scan terminates.

[0051] The density constraint is determined as follows: after considering the current candidate MAC address, the density value under the current expansion pattern is calculated. The numerator of the density calculation formula is the number of MAC addresses already included in the current cluster minus 1, and the denominator is the integer difference between the integer value of the current candidate MAC address and the integer value of the starting MAC address. Subtracting 1 from the numerator ensures that the density value is 0 when the number of included MAC addresses is only 1, and the density gradually increases as the number of included MAC addresses increases, reflecting the density of the distribution of identified MAC addresses within the interval. When the calculated density value is lower than the preset minimum density threshold, it indicates that the identified MAC addresses within the current interval are too sparse. Continuing to expand will include a large number of address gaps lacking the support of identified MAC addresses. The model attribution of unidentified MAC addresses in these gaps lacks sufficient basis, and the reliability of the inference will decrease, triggering the density constraint and terminating the scan. If the density value is not lower than the preset threshold, the density constraint is passed, the candidate MAC address is officially included in the current cluster, the number of included MAC addresses increases by 1, and the scan continues to the next candidate MAC address.

[0052] The checks for the above three constraints are performed sequentially during the scan, in the order of adjacent spacing constraint, model purity constraint, and density constraint. Once any of these constraints is triggered, the scan stops immediately, and no further candidate MACs are processed.

[0053] After the scan stops, the endpoint MAC address is selected from the candidate MAC addresses that have passed all constraint checks and been formally included in the cluster before the stop, based on interval density and interval coverage. This is not simply selecting the last included candidate MAC address as the endpoint; instead, every candidate MAC address included before the stop is considered as a candidate endpoint for comprehensive evaluation. For each candidate endpoint, a comprehensive score is calculated based on the interval density and interval coverage corresponding to that point, and the candidate endpoint with the highest comprehensive score is selected as the endpoint MAC address. Interval density is represented by the density value corresponding to the candidate endpoint, and interval coverage is represented by the integer difference between the candidate endpoint and the starting point. The comprehensive score takes both dimensions into account, ensuring that the generated cluster has sufficient density to support its confidence while extending its coverage as much as possible to encompass more unidentified MAC addresses. After determining the endpoint MAC address, the processing process outputs a cluster record, which includes the starting MAC address, the endpoint MAC address, and the terminal device model identifier corresponding to the starting point.

[0054] After this round of clustering is completed, the processing process updates the scan position pointer to the position of the next record of the selected endpoint MAC address in the original ordered MAC address sequence. If a next record exists, the state variable is reset and the next round of clustering begins, using that record as the new starting point; if the end of the sequence has been reached, the clustering task for this data shard is complete. The cluster records generated by each round of clustering are stored one by one in the aggregation result list of the processing process, collectively forming the cluster set corresponding to the respective data shard.

[0055] In one or more embodiments of the present invention, selecting the endpoint MAC address based on the interval density and interval coverage includes: Among the candidate MACs scanned before the scan stops, each candidate MAC is used as a candidate termination point; For each candidate termination point, a comprehensive score is calculated based on the density of the interval and the coverage of the interval corresponding to the candidate termination point; The candidate termination point with the highest overall score is selected as the endpoint MAC address; The overall score is calculated as follows: score=α×density+β×log(distance+1) Where α is the density weight, β is the span weight, density is the interval density corresponding to the candidate termination point, and distance is the integer difference between the candidate termination point and the starting point.

[0056] After the scan stops, multi-constraint greedy clustering faces the problem of determining the endpoint of the cluster from the candidate MACs that have passed the constraint checks. Simply selecting the last included candidate MAC as the endpoint ignores the dynamic characteristics of interval quality changing with span during the expansion process, and may miss better termination positions. This embodiment comprehensively evaluates each candidate MAC within the expansion window and selects the candidate MAC with the highest comprehensive score as the endpoint, so that the generated clusters achieve a balance between interval density and interval coverage.

[0057] Specifically, during the scanning process, each candidate MAC address included, along with the starting point, constitutes a potential aggregation interval. This interval has corresponding density and span values. The density value reflects the density of the identified MAC addresses within the interval, while the span value reflects the coverage area of ​​the interval in the MAC address space. As the scan progresses, the number of included candidate MAC addresses gradually increases, and the span gradually expands, but the density may decrease due to the inclusion of MAC addresses from sparse regions. Therefore, different candidate termination points correspond to different combinations of density and span, forming a set of feasible solutions within the legal expansion window.

[0058] After the scan stops, each candidate MAC that has passed all constraint checks and has been formally included in the cluster before the stop is treated as a candidate termination point. For each candidate termination point, values ​​are extracted from the interval density and span data recorded during the clustering process to calculate a comprehensive score. The comprehensive score is calculated as follows: score = α × density + β × log(distance + 1), where density is the interval density corresponding to the candidate termination point, i.e., the ratio of the number of included MACs to the integer difference between the candidate termination point and the starting point; distance is the integer difference between the candidate termination point and the starting point; α is the density weight, and β is the span weight, both of which are preset positive real number parameters used to adjust the relative importance of the density and span terms in the comprehensive score.

[0059] The scoring function is designed to balance the density and coverage of the intervals. The density term α×density is directly proportional to the interval density, encouraging the selection of intervals with densely distributed identified MAC addresses as clusters to improve the consistency and reliability of device models within the clusters. The span term β×log(distance+1) increases logarithmically with the span, encouraging the extension of the interval coverage as much as possible within the limits of density to encompass more unidentified MAC addresses and improve the coverage of subsequent model inferences. The choice of the logarithmic function makes the marginal benefit of the span term decrease as the span increases. That is, when the interval is already large enough, the scoring gain from further expansion gradually decreases, avoiding blindly pursuing a large span at the expense of density. The addition of one to distance ensures that when the candidate termination point is the starting point itself, the logarithmic term takes the value of log(1), which is 0. The span term does not contribute to the scoring, and the scoring is determined only by the density term, which is consistent with the intuitive understanding that the starting point itself does not constitute an effective span.

[0060] The values ​​of density weight α and span weight β determine the preference direction of clusters between density and coverage. When α is relatively large compared to β, the overall score focuses more on interval density, and the generated clusters tend to be denser but may have smaller spans. When β is relatively large compared to α, the overall score focuses more on coverage, and the generated clusters tend to have larger spans but are less sensitive to density. In actual deployment, the values ​​of α and β can be set according to the distribution characteristics of terminal device MAC addresses and the service's need for a trade-off between inferred coverage and accuracy, or they can be determined through grid search or empirical tuning on labeled datasets.

[0061] After calculating the overall score of each candidate termination point, the scores of all candidate termination points are compared, and the candidate termination point with the highest overall score is selected as the endpoint MAC address of the current aggregation cluster. If multiple candidate termination points have the same overall score and are all the highest values, the candidate termination point with the largest span can be selected as the endpoint to prioritize coverage of a wider address range under the same score. After determining the endpoint, the processing process outputs an aggregation cluster record, which includes the starting MAC address, the endpoint MAC address, and the terminal device model identifier corresponding to the starting point.

[0062] In one or more embodiments of the present invention, a machine learning classification model is used to perform quality evaluation on the initial aggregated clusters, and aggregated clusters that meet preset quality conditions are retained, including: For each cluster in the initial cluster, a five-dimensional feature vector is extracted. The five-dimensional feature vector includes: the terminal device model identifier corresponding to the cluster, the number of aggregated MAC addresses in the cluster, the cluster density, the cluster span, and the cluster's overall score. The density is the ratio of the number of aggregated MAC addresses minus one to the integer difference between the cluster's endpoint MAC address and its starting point MAC address. The span is the integer difference between the cluster's endpoint MAC address and its starting point MAC address plus one. The five-dimensional feature vector is input into the machine learning classification model, and the machine learning classification model outputs a quality prediction label. The quality prediction label is used to indicate whether the cluster belongs to a high-quality cluster or a low-quality cluster. Clusters whose quality prediction labels indicate high quality are retained, while clusters whose quality prediction labels indicate low quality are removed.

[0063] A machine learning classification model is used to assess the quality of the initial clusters, automatically selecting clusters with high confidence from a large number of clustering results, replacing manual review of each cluster, and achieving standardization and scalability of quality assessment. The initial cluster set is generated by the aforementioned parallel greedy clustering stage. It includes high-quality clusters that truly reflect the distribution pattern of MAC addresses of terminal device models, but may also contain low-quality clusters caused by data noise, random factors, or boundary effects of clustering parameters. These low-quality clusters need to undergo quality filtering before entering the subsequent interval merging and model inference stages.

[0064] The quality assessment targets each cluster in the initial cluster set. For each cluster, a five-dimensional feature vector is first extracted, with each feature characterizing the cluster's credibility from different perspectives.

[0065] The first feature is the terminal device model identifier corresponding to the cluster. This feature identifies the terminal device model to which the cluster belongs, providing contextual information for quality assessment—different models have different distribution patterns in the MAC address space; some models have more concentrated and continuous address allocations, while others have more dispersed allocations. Therefore, the model identifier itself has reference value for judging the quality of the cluster. In the feature vector, the terminal device model identifier is processed as a categorical feature.

[0066] The second feature is the number of aggregated MAC addresses within the cluster, which is the total number of MAC addresses associated with terminal device model identifiers included in the clustering process. This number directly reflects the sample support strength of the cluster: the more identified MAC addresses included, the more sufficient labeled samples support the cluster, and the higher its credibility; conversely, if the number of aggregated MAC addresses is too small, the cluster may consist of only a small number of randomly occurring MAC addresses, lacking statistically significant support.

[0067] The third feature is the density of clusters, calculated as the ratio of the number of aggregated MAC addresses minus one to the integer difference between the starting and ending MAC addresses of the cluster. Density reflects the density of identified MAC addresses within the MAC address span covered by the cluster. High-density clusters indicate a dense distribution of identified MAC addresses within that address range, with smaller gaps for unidentified MAC addresses, making it easier to infer the model of the unidentified MAC addresses within these gaps. Low-density clusters indicate the presence of many gaps for unidentified MAC addresses, which may belong to other models, leading to uncertainty in the inference.

[0068] The fourth feature is the span of the cluster, calculated as the integer difference between the starting and ending MAC addresses of the cluster plus one. The span reflects the coverage area of ​​the cluster in the MAC address space. Clusters with larger spans can cover more address points and have higher inferred coverage value; however, span alone is insufficient to judge quality and must be considered in conjunction with density and the number of clusters already aggregated.

[0069] The fifth feature is the comprehensive score of the cluster. This score is calculated by the comprehensive scoring function during the clustering stage. It takes into account both the density of intervals and the coverage of intervals, and is a comprehensive quantitative indicator of the overall performance of the cluster during the clustering process.

[0070] The five dimensions of features described above collectively characterize the overall quality of a cluster from five perspectives: model context, sample size, distribution density, coverage, and historical ratings. After feature extraction, the five-dimensional feature vector is input into a pre-trained machine learning classification model. This model is trained offline using a training sample set constructed from historical clustering results. The training sample set is constructed by sampling records from clusters generated in previous clustering sessions, extracting the aforementioned five-dimensional feature vector from each record, and then manually or semi-automatically labeling them with quality tags. Quality tags indicate whether a cluster is high-quality or low-quality, and the labeling criteria may include the known distribution patterns of device manufacturers within the cluster, the cluster's accuracy in predicting reserved verification samples, and domain expert experience.

[0071] Machine learning classification models take a five-dimensional feature vector as input, perform internal calculations, and output a quality prediction label indicating whether the cluster belongs to a high-quality cluster or a low-quality cluster. The model's internal computational logic is determined by the mapping relationship between features and quality labels learned from the training sample set during the training phase.

[0072] Based on the quality prediction labels output by the model, the initial cluster set is filtered: clusters whose quality prediction labels indicate high quality are retained and included in the subsequent processing flow; clusters whose quality prediction labels indicate low quality are removed and will not proceed to the interval merging and model inference stages.

[0073] Through the aforementioned quality assessment and filtering, low-quality clusters in the initial cluster set are automatically identified and removed, while the remaining clusters have higher confidence levels. As new device models constantly emerge in the terminal device market, the distribution of terminal device model identifiers changes over time. Therefore, the machine learning classification model can be periodically retrained and updated. During updates, the recently accumulated aggregation results are used to resample and construct the training sample set, retraining the model parameters to maintain the model's adaptability to the current data distribution and ensure the continued accuracy of the quality assessment.

[0074] In one or more embodiments of the present invention, the machine learning classification model is a random forest model, and the training process of the random forest model includes: Several cluster records are sampled from the historical aggregation results. A five-dimensional feature vector is extracted from each cluster record, and quality labels are labeled to form a training sample set. The quality labels include high-quality labels and low-quality labels. Using a five-dimensional feature vector as input and quality labels as output, random search cross-validation is used to search for the hyperparameters of the random forest model. The hyperparameters include the number of decision trees, maximum depth, and minimum number of sample splits. During cross-validation, hyperparameter combinations that meet preset conditions for classification performance metrics on the validation set are selected to obtain the trained random forest model, and the trained random forest model is persistently saved.

[0075] The training of the random forest model is completed offline. The training process does not rely on the real-time clustering process and can be executed independently and updated periodically. The first step in training is to construct a training sample set. A number of cluster records are sampled from historical clustering results. The number of samples is sufficient to ensure that the training sample set statistically reflects the quality distribution of clusters. During sampling, the proportion of high-quality and low-quality clusters is considered to ensure that the number of samples of the two classes in the training set is roughly balanced, avoiding class imbalance that could lead to the model biasing towards the majority class. For each sampled cluster record, a five-dimensional feature vector is extracted. The composition of the feature vector is completely consistent with that in the quality assessment stage, including the terminal device model identifier corresponding to the cluster, the number of aggregated MAC addresses within the cluster, the cluster density, the cluster span, and the cluster's overall score. Each feature vector needs to be labeled with a quality label, including both high-quality and low-quality labels. Labeling can be done manually based on the known distribution patterns of devices within the cluster and domain experience, or it can be done semi-automatically. For example, by reserving a validation dataset to evaluate the inference accuracy of the clusters, clusters with inference accuracy above a set threshold are labeled as high-quality, and those below the threshold are labeled as low-quality. After feature extraction and labeling, a structured training sample set is formed, with each row corresponding to one cluster sample, containing five feature columns and one label column.

[0076] After the training sample set is constructed, the model training and hyperparameter search phase begins. A random forest classifier is trained using the five-dimensional feature vectors from the training sample set as input and the quality labels as output. The random forest model involves multiple hyperparameters, which are preset before model training begins, and their values ​​directly affect the model's classification performance. The number of decision trees determines the number of weak classifiers; too few trees may lead to underfitting, while too many increase training time and diminishing marginal returns. The maximum depth limits the maximum number of layers each decision tree can grow; excessive depth increases the risk of overfitting, while insufficient depth makes it difficult for the model to learn the complex relationship between features and labels. The minimum number of splits specifies the minimum number of samples required for a decision tree node to continue splitting; this parameter also controls overfitting, with a higher value indicating a more conservative model.

[0077] To determine the optimal combination of the aforementioned hyperparameters, a random search cross-validation method is used for hyperparameter search. The random search randomly samples several sets of hyperparameter combinations from a predefined hyperparameter value space. Each set includes a specific value for the number of decision trees, maximum depth, and minimum number of sample splits. For each sampled hyperparameter combination, cross-validation is performed on the random forest model under those parameters: the training sample set is divided into several subsets, and one subset is used as training data while another is used as validation data, repeated multiple times. The average classification performance metric on the validation set is calculated as the performance evaluation result of that hyperparameter combination. Classification performance metrics can include accuracy, the harmonic mean of precision and recall, or the area under the receiver operating characteristic curve (AUC), etc. The specific metric can be selected based on the business's tolerance for false positives and false negatives. After the random search traverses a predefined number of hyperparameter combinations, the set whose classification performance metric on the validation set meets the predefined conditions is selected as the final hyperparameter combination from all evaluated hyperparameter combinations. The preset conditions can be either the highest performance index among all evaluated combinations, or a combination whose performance index reaches a preset threshold and whose model complexity is low.

[0078] Using the selected hyperparameter combination, the random forest model is retrained on the entire training sample set to obtain the trained model. After training, the model's structural parameters, the node splitting rules of each decision tree, and related metadata are persistently saved to a storage medium. The saving format can be a standard model file format supported by the training framework. In the subsequent quality evaluation stage, this persistently saved model file can be loaded to perform quality predictions on new cluster samples without retraining.

[0079] Since the distribution of terminal device models in a home network changes over time, the appearance of new device models may lead to a decline in the performance of historical models. To address this, a model update process can be triggered periodically: new aggregation results accumulated over a period of time are used as new samples, which are then merged with historical training samples or the training sample set is reconstructed according to time decay weights. Hyperparameter search and model training are then re-executed, and the updated model replaces the old model, ensuring that the quality assessment remains adaptable to the current data distribution.

[0080] In one or more embodiments of the present invention, clusters of terminal devices with the same model identifier and adjacent MAC address spaces are merged when their density after merging meets a preset condition, including: The remaining clusters are arranged in ascending order of their starting MAC addresses to form a sequence of clusters to be merged. Traverse the sequence of clusters to be merged. For two adjacent clusters with the same terminal device model identifier, calculate the new interval density after merging the two adjacent clusters. When the density of the new interval after merging is not lower than the preset merging density threshold, two adjacent clusters are merged into one cluster. The merged cluster is identified by the starting MAC address of the first cluster of the two adjacent clusters, the ending MAC address of the second cluster of the two adjacent clusters, and the terminal device model identifier. When the density of the new merged interval is lower than the preset merge density threshold, two adjacent clusters are retained as independent clusters. The calculation of the new interval density after merging two adjacent clusters includes: Obtain the number of aggregated MAC addresses in the previous cluster and the number of aggregated MAC addresses in the next cluster, and calculate the total number of aggregated MAC addresses after merging, which is the sum of the two aggregated MAC address counts. Obtain the integer difference between the endpoint MAC address of the next aggregation cluster and the starting MAC address of the previous aggregation cluster, and calculate the number of MAC addresses covered by the new interval after merging as the integer difference plus one; The ratio of the total number of aggregated MAC addresses to the number of MAC addresses covered by the new merged interval is calculated and used as the density of the new merged interval.

[0081] Because each data shard during the clustering phase is processed independently by different processes, multiple clusters with the same terminal device model identifier and adjacent MAC address spaces may be generated near the shard boundaries. For example, the last cluster generated by clustering within a shard and the first cluster generated by clustering in an adjacent shard may be adjacent in address space and belong to the same terminal device model. Furthermore, within the same shard, different rounds of clustering may split a previously continuous address segment into multiple adjacent smaller segments due to differences in the timing of constraint triggering during the expansion process. These adjacent clusters with the same model actually constitute a larger continuous interval in address space. If retained as independent entries, this would increase the redundancy of the final cluster table and reduce the efficiency of subsequent interval matching queries. By merging adjacent intervals, these fragmented clusters with the same model can be merged into a larger continuous interval, reducing the number of feature library entries while maintaining or improving interval quality.

[0082] The merge operation first sorts all the clusters retained after quality assessment in ascending order by their starting MAC address, forming a sequence of clusters to be merged. This ascending order ensures that clusters that are adjacent in the MAC address space are also adjacent in the sequence, facilitating the merging of all adjacent clusters in a single traversal.

[0083] When traversing the sequence of aggregated clusters to be merged, a merge cursor is maintained, initially pointing to the first aggregated cluster in the sequence. Using the aggregated cluster pointed to by the merge cursor as a reference, its relationship with the next aggregated cluster in the sequence is checked. The first condition checked is whether the terminal device model identifiers of the two aggregated clusters are the same. If the terminal device model identifiers are different, it means that the two aggregated clusters belong to different device models, and their adjacency in the address space is a natural boundary between address segments of different models; therefore, they should not be merged, and the merge cursor is moved to the next aggregated cluster to continue traversal. If the terminal device model identifiers are the same, the density of the new interval after merging the two aggregated clusters is further calculated to determine whether merging would lead to a decrease in interval quality.

[0084] The specific steps for calculating the density of the new merged interval are as follows: First, obtain the number of aggregated MAC addresses in the previous cluster, denoted as the first number; second, obtain the number of aggregated MAC addresses in the next cluster, denoted as the second number. Add the first number and the second number to obtain the total number of aggregated MAC addresses after merging, representing the total number of MAC addresses actually associated with terminal device model identifiers within the new merged interval. Next, obtain the endpoint MAC address of the next cluster and the starting point MAC address of the previous cluster, and calculate the integer difference between them. Since the number of address points covered from the starting point to the endpoint in the MAC address space is equal to the endpoint minus the starting point plus one, the number of MAC addresses covered by the new merged interval is the integer difference plus one. Divide the total number of aggregated MAC addresses after merging by the number of MAC addresses covered by the new merged interval; the resulting ratio is the density of the new merged interval.

[0085] The density calculation method maintains the same mathematical logic as the density definition in the clustering stage. The numerator is the number of identified MACs in the interval, and the denominator is the total number of address points covered by the interval. The density value reflects the density of the distribution of identified MACs in the merged interval.

[0086] After calculating the density of the merged new interval, it is compared with a preset merge density threshold. The preset merge density threshold is a quality threshold for determining whether adjacent clusters of the same model should be merged. When the density of the merged new interval is not lower than the preset merge density threshold, it indicates that the merged interval still has sufficient density support, and the gap between the two adjacent clusters does not significantly dilute the overall interval quality, so the merge operation is performed. The merged cluster is defined by the starting MAC address of the first cluster of the two adjacent clusters, the ending MAC address of the second cluster of the two adjacent clusters, and the common terminal device model identifier. The number of aggregated MAC addresses is updated to the total number of aggregated MAC addresses after the merge. After the merge is completed, the merge cursor remains stationary, the merged second cluster in the sequence is removed, and the merged cluster is used as the new benchmark to continue merging with the next cluster in the sequence. This ensures that in scenarios with multiple consecutive adjacent clusters of the same model, they can be merged into a single overall interval step by step.

[0087] When the density of the merged new interval is lower than the preset merge density threshold, it indicates that the gap between the two clusters is too large or the identified MAC distributions within each cluster are insufficient to support the overall density of the merged interval. Merging will lead to a significant decrease in interval quality, so merging is not performed. The merge cursor moves to the next cluster, using it as the new baseline, and continues traversing.

[0088] The traversal process continues until the merging cursor reaches the end of the sequence or there are no more adjacent clusters to check. After the traversal is complete, the clusters remaining in the sequence are the merged clusters. The merged clusters are then sorted in ascending order by their starting MAC addresses to form the final cluster table. Each record in the final cluster table defines a continuous coverage interval in the MAC address space for a terminal device model identifier. This interval will serve as the basis for subsequent interval matching and model inference of the MAC address to be inferred.

[0089] In one or more embodiments of the present invention, an ordered index is constructed based on the starting MAC address of the aggregated cluster in the final aggregated cluster table. Range matching is performed on the MAC address to be predicted that is not associated with a terminal device model identifier to predict the terminal device model identifier corresponding to the MAC address to be predicted, including: Sort the final aggregated cluster table in ascending order by the starting MAC address; Extract the starting MAC address of each cluster from the final cluster table after sorting to form an ordered array of starting MAC addresses; The endpoint MAC address and terminal device model identifier of each cluster are arranged in the same order as the ordered starting MAC address array to form the endpoint MAC address array and the terminal device model identifier array, respectively. Convert the MAC address to be inferred into an integer value; Perform a binary search on the ordered array of starting MAC addresses to locate the target index, which is the maximum index value that satisfies the condition that the starting MAC address does not exceed the MAC address to be predicted. Read the endpoint MAC address and terminal device model identifier corresponding to the target index; When the MAC address to be predicted is less than or equal to the destination MAC address, the terminal device model identifier of the MAC address to be predicted is predicted as the terminal device model identifier corresponding to the target index. When the MAC address to be inferred is greater than the endpoint MAC address, the MAC address to be inferred is marked as unmatched.

[0090] The final cluster table is a collection of clusters formed after multi-constraint greedy clustering, quality assessment and filtering, and merging of adjacent intervals. Each cluster record in the table contains three fields: the starting MAC address, the ending MAC address, and the terminal device model identifier, defining a continuous coverage interval of the terminal device model identifier in the MAC address space. Since the cluster table has already been sorted in ascending order by the starting MAC address during the merging phase, the index construction here can directly use this sorting order.

[0091] The index construction process is as follows: From the final aggregated cluster table, the starting MAC address of each aggregated cluster is extracted sequentially. The extracted starting MAC addresses are then stored in an array to form an ordered starting MAC address array. The elements in this array are strictly ordered in ascending order of integer values, and any two adjacent elements satisfy the condition that the value of the preceding element is less than the value of the following element. Simultaneously, the ending MAC address and terminal device model identifier of each aggregated cluster are stored in the ending MAC address array and the terminal device model identifier array, respectively, in the exact same order as the ordered starting MAC address array. The indices of the three arrays correspond one-to-one: the element at index i in the ordered starting MAC address array, the element at index i in the ending MAC address array, and the element at index i in the terminal device model identifier array together describe the three fields of the i-th aggregated cluster in the final aggregated cluster table. This index structure stores the three fields of the cluster table by column, so that binary search only needs to compare values ​​on the starting MAC address array. After the search is successful, the corresponding ending MAC address and terminal device model identifier can be read directly from the other two arrays using the same index, without having to look up the original cluster table.

[0092] For a MAC address to be inferred that is not associated with a terminal device model identifier, it is first converted into a 48-bit unsigned integer according to the same standardization rules as in the input data preprocessing stage. Then, a binary search is performed on the ordered starting MAC address array to determine the cluster interval to which the MAC address to be inferred belongs in the MAC address space.

[0093] The specific process of binary search is as follows: The left boundary of the search range is the starting position of the ordered MAC address array, and the right boundary is the ending position of the array. In each round of search, the midpoint of the current search range is calculated, and the integer value of the starting MAC address at the midpoint is read and compared with the integer value of the MAC address to be predicted. If the integer value of the MAC address to be predicted is less than the integer value of the starting MAC address, it means that the MAC address to be predicted is located in the interval to the left of the midpoint, and the right boundary of the search range is shrunk to the midpoint minus one. If the integer value of the MAC address to be predicted is greater than or equal to the integer value of the starting MAC address, it means that the MAC address to be predicted may be located within the interval starting from the midpoint or to the right of it, and the left boundary of the search range is shrunk to the midpoint. This halving and shrinking of the search range is repeated until the left and right boundaries coincide or intersect, at which point the search ends. The resulting location index is the largest index value in the ordered MAC address array that satisfies the condition that the starting MAC address does not exceed the MAC address to be predicted, and is called the target index. The cluster corresponding to the target index is the last cluster in the address space whose starting point is before or coincides with the MAC address to be inferred, and is the only possible belonging range of the MAC address to be inferred.

[0094] After the binary search is completed, the corresponding endpoint MAC address is read from the endpoint MAC address array using the target index, and the corresponding terminal device model identifier is read from the terminal device model identifier array. It is then determined whether the integer value of the MAC address to be inferred is less than or equal to the read endpoint MAC address. If this condition is met, it means that the MAC address to be inferred falls within the closed interval defined by the start and end MAC addresses of the cluster in the numerical space. The address range covered by the cluster has interpretable model attribution for the MAC address to be inferred, and the terminal device model identifier of the MAC address to be inferred is inferred as the terminal device model identifier corresponding to the cluster.

[0095] If the integer value of the MAC address to be inferred is greater than the endpoint MAC address read, it means that although the MAC address to be inferred has exceeded the last starting point but not the endpoint of its cluster, it has not yet reached the starting point of the next cluster and is in the gap between two clusters, not falling within the coverage of any cluster. Since the target index is already the largest index that satisfies the condition that the starting point does not exceed the MAC address to be inferred, there are no other clusters that may cover this address, and the MAC address to be inferred is marked as unmatched.

[0096] In batch inference scenarios, the binary search and interval judgment processes described above are executed independently for each MAC address to be inferred. Since the time complexity of binary search is logarithmic, the search time increases slowly with the total number of clusters. Even if the final cluster table contains hundreds of thousands of cluster records, the number of comparisons required for a single search can be kept within a small range, meeting the performance requirements for efficient batch inference of large-scale sets of MAC addresses. The inference results can be written in batches to a result table, recording the original identifier of each MAC address to be inferred, the inferred terminal device model identifier, or a non-matching marker, for subsequent completion and statistical analysis of home network device profiles.

[0097] In one or more embodiments of the present invention, after the final aggregated cluster table is formed, the final aggregated cluster table is encrypted and packaged into a feature library binary file, and deployed to the home network device identification system. This enables the identification system to infer the model of the received terminal MAC address and complete the home network device profile. This step transforms the aggregated cluster data generated in the offline clustering analysis stage into a standardized feature library that can be directly called by the online identification system, achieving a technical closed loop from data analysis to business application.

[0098] Each cluster record in the final cluster table contains three fields: the starting MAC address, the ending MAC address, and the terminal device model identifier, defining a continuous coverage range of a terminal device model in the MAC address space. Before packaging to generate the feature library binary file, the final cluster table is first converted to a standard range format. The conversion method is to use the starting MAC address in the cluster record as the range start address, the ending MAC address as the range end address, and the terminal device model identifier as the device model corresponding to that range. Each cluster record corresponds to one standard range entry. If the cluster table also contains auxiliary fields generated during the clustering process, such as the number and density of aggregated MAC addresses, these are retained during the conversion or selectively retained according to business needs to enrich the information dimensions of the feature library.

[0099] The aggregated cluster data, converted to a standard range format, is written to the feature library binary file. The internal structure of the binary file can employ a custom compact storage format to reduce file size and parsing overhead during loading. One optional storage structure is as follows: metadata information, including the feature library version number, the total number of aggregated cluster entries, the file creation timestamp, and a checksum, is written to the file header for version management and integrity verification; the file body sequentially stores each range entry, with each entry containing the starting MAC address, the ending MAC address, and the terminal device model identifier, arranged sequentially in fixed-byte lengths. Both the starting and ending MAC addresses are stored as 48-bit unsigned integers in binary form, and the terminal device model identifier is stored as an encoded numerical value to save storage space.

[0100] To enhance the security of the signature database during transmission and deployment, and to prevent unauthorized third parties from stealing or tampering with the data, the signature database binary file is encrypted and packaged. A symmetric encryption algorithm can be used, and the encryption key is securely managed by the deployment system. The encryption packaging tool receives the plaintext signature database binary file and the encryption key as input, and outputs an encrypted signature database binary file. Encryption can be performed offline; the encrypted file remains in ciphertext form during transmission and storage, and is only decrypted and used in memory when the identification system loads it.

[0101] After encryption and packaging, the signature database binary file is deployed to the home network device identification system. Deployment can be done by distributing the signature database file to a designated directory on the server hosting the identification system, allowing the system to automatically load the new version of the signature database upon startup or upon receiving an update command. When loading the signature database file, the system first decrypts the file using the corresponding decryption key, verifies the version number and checksum in the file header metadata, and confirms file integrity and version validity. Then, it parses and loads the range entries in the file body into memory, constructing an ordered index structure for subsequent real-time model inference of the terminal's MAC address. The loading process supports a hot update mechanism, meaning that after the new version of the signature database is loaded in the background, the system smoothly switches to the new version index without interrupting online services.

[0102] After loading the feature library, the identification system, upon receiving the terminal MAC address reported by the home gateway, performs model inference using the aforementioned binary interval matching method. For terminal devices that have reported fingerprint features, the identification system can perform cross-validation by combining fingerprint matching results and interval inference results to improve identification accuracy. For terminal devices that have not reported fingerprint features, the identification system directly assigns a model identifier to the terminal device based on the interval inference result. After inference, the identification system associates and stores the inferred terminal device model identifier with the MAC address, and combines it with device information provided by other data sources to output profile attributes including device model, manufacturer, and device type, thus completing the profile information of the home network devices. The completed device profile can be used by the operation and maintenance system or business system, for example, for home network topology display, device management and fault diagnosis, and fine-grained network policy control based on device type.

[0103] As new device models continue to be launched in the terminal device market and the number of identified MAC address samples continues to accumulate, the final aggregated cluster table needs to be updated periodically to maintain the inferred coverage and accuracy. The update process can be set to be triggered on a timed basis: reload the latest MAC address data associated with terminal device model identifiers and active MAC address data not associated with terminal device model identifiers from the original big data storage system, re-execute the entire process of data sharding preprocessing, parallel multi-constraint greedy clustering, machine learning quality assessment, adjacent interval merging, and feature library packaging and deployment, generate a new version of the feature library binary file and deploy it online, completing the iterative update of the feature library.

[0104] This invention also provides a MAC address intelligent aggregation system for profiling home network devices, comprising: The data fragmentation preprocessing module is used to obtain the MAC address associated with the terminal device model identifier, divide the MAC address into multiple data fragments according to the manufacturer prefix, sort the MAC addresses in each data fragment by integer value, and obtain an ordered MAC address sequence. The parallel clustering module is used to perform multi-constraint greedy clustering on ordered MAC address sequences in a parallel processing manner to obtain preliminary clusters. The multi-constraint greedy clustering defines the expansion window based on adjacent distance constraints, model purity constraints, and density constraints, and determines the termination point based on the interval density and interval coverage to form clusters. The intelligent quality assessment module is used to perform quality assessment on the initial clusters using a machine learning classification model, and retain the clusters that meet the preset quality conditions. The interval merging module is used to merge clusters of terminal devices with the same model identifier and adjacent MAC address spaces when the density after merging meets the preset conditions, to form the final cluster table. The inference matching module is used to construct an ordered index based on the starting MAC address of the aggregated cluster in the final aggregated cluster table, perform range matching on the MAC address to be inferred that is not associated with the terminal device model identifier, and infer the terminal device model identifier corresponding to the MAC address to be inferred.

[0105] In one or more embodiments of the present invention, the system further includes: a feature library deployment and profiling application module, used to encrypt and package the final aggregated cluster table into a feature library binary file, deploy it to the home network device identification system, so that the identification system can perform model inference on the received terminal MAC address and complete the home network device profile.

[0106] This invention employs a multi-constraint greedy clustering mechanism, simultaneously applying adjacent spacing constraints, model purity constraints, and density constraints during the aggregation process. These three constraints jointly define the expansion window, and the termination point is determined within the expansion window based on the density and coverage of the intervals. In principle, this avoids the problems of heterogeneous model contamination and interval fragmentation caused by improper threshold settings in a single threshold scheme. This makes the MAC address intervals generated by aggregation more closely match the actual address allocation rules of terminal equipment manufacturers, thereby improving the accuracy of model inference for unreported fingerprint terminal equipment. This invention also introduces a machine learning classification model to automatically assess the quality of the initial aggregated clusters, and objectively screens the aggregation results based on the multidimensional features of the aggregated clusters, replacing the manual sampling and review process. This standardizes and makes the quality assessment process of the feature library reproducible, supports high-frequency iterative updates of the feature library, and reduces human maintenance costs. This invention also segments the MAC address space by manufacturer prefix, leveraging the inherent contiguous nature of MAC addresses from the same manufacturer to divide global data into independent subspaces. Combined with parallel processing, clustering tasks are performed synchronously on each segment, reducing the processing time for massive MAC address data as the parallel scale expands, thus meeting the large-scale processing needs of billions of terminal data points. Simultaneously, an ordered index is constructed based on the starting MAC address of the final aggregated cluster table, and binary interval matching is performed on the MAC addresses to be inferred. The time complexity of a single inference is logarithmic, efficiently supporting batch model inference for hundreds of millions of terminal devices, thereby improving the identification coverage and completeness of home network device profiles.

[0107] The above provides a detailed description of the intelligent MAC address aggregation method and system for profiling home network devices. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the invention. Therefore, the content of this specification should not be construed as a limitation of the invention.

Claims

1. A method for intelligent aggregation of MAC addresses for profiling home network devices, characterized in that, The method includes: Obtain the MAC address associated with the terminal device model identifier; The MAC addresses are divided into several data segments according to the manufacturer prefix, and the MAC addresses in each data segment are sorted by integer values ​​to obtain an ordered MAC address sequence. A parallel processing approach is used to perform multi-constraint greedy clustering on the ordered MAC address sequence to obtain preliminary clusters. The multi-constraint greedy clustering defines the expansion window based on adjacent spacing constraints, model purity constraints, and density constraints, and determines the termination point based on the interval density and interval coverage to form clusters. The quality of the initial clusters is evaluated using a machine learning classification model, and clusters that meet the preset quality conditions are retained. For clusters of terminal devices with the same model identifier and adjacent MAC address spaces, they are merged when the density after merging meets the preset conditions to form the final cluster table. An ordered index is constructed based on the starting MAC address of the aggregated cluster in the final aggregated cluster table. Range matching is performed on the MAC address to be inferred that is not associated with the terminal device model identifier, and the terminal device model identifier corresponding to the MAC address to be inferred is inferred.

2. The method according to claim 1, characterized in that, The step of dividing the MAC address into multiple data fragments according to the manufacturer prefix, and sorting the MAC addresses within each data fragment by integer value, includes: The MAC addresses are standardized by converting them into a uniform 12-digit hexadecimal string with no separators in uppercase, and MAC address records containing illegal characters or invalid formats are removed. The MAC address, after the standardization preprocessing, is resolved into a forty-eight-bit unsigned integer. Extract the high 24 bits of the 48-bit unsigned integer as the manufacturer prefix; Based on the manufacturer prefix, MAC addresses with the same manufacturer prefix are grouped into the same data segment, and different manufacturer prefixes correspond to different data segments. Each data segment carries a non-overlapping MAC address subspace. For each data segment, the MAC addresses are sorted in ascending order according to their corresponding 48-bit unsigned integer values, and the sorted MAC addresses are deduplicated to obtain the ordered MAC address sequence.

3. The method according to claim 2, characterized in that, The parallel processing method involves performing multi-constraint greedy clustering on the ordered MAC address sequence to obtain preliminary clusters, including: Each data slice is assigned an independent processing process, and multiple processing processes are executed in parallel, with each processing process being independent of the others. Each processing process independently performs multi-constraint greedy clustering on the ordered MAC address sequence corresponding to its data shard, generating a set of aggregated clusters corresponding to its data shard. After all processing processes are completed, the aggregated clusters corresponding to each data shard are merged. The clusters are deduplicated using the triplet of the starting MAC address, ending MAC address, and terminal device model identifier as the deduplication key, and then sorted in ascending order by the starting MAC address to form the initial aggregated clusters.

4. The method according to claim 3, characterized in that, In each processing step, the multi-constraint greedy clustering generates each cluster in the set of clusters corresponding to the data shard by repeatedly executing the following steps: Starting from the MAC address at the current position in the ordered MAC address sequence, record the terminal device model identifier corresponding to the starting point, and record the number of MAC addresses that have been included in the current cluster and the number of different MAC addresses that have been skipped during the scanning process; Starting from the starting point, scan the candidate MAC addresses one by one along the direction of increasing integer values ​​in the ordered MAC address sequence; During the scanning process, when the integer difference between the candidate MAC and the previous MAC exceeds the preset maximum adjacent distance, the adjacent distance constraint termination is triggered; when the terminal device model identifier of the candidate MAC is different from the terminal device model identifier corresponding to the starting point, the number of skipped different model MACs is increased by one; when the increased number of skipped different model MACs exceeds the product of the number of MACs already included in the current cluster and the preset maximum skip ratio, the model purity constraint termination is triggered; when the density calculated based on the ratio of the number of MACs already included in the current cluster minus one to the integer difference between the candidate MAC and the starting point is lower than the preset minimum density threshold, the density constraint termination is triggered. The scan stops when any of the constraints is triggered. Among the candidate MAC addresses scanned before the scanning stops, the endpoint MAC address is selected based on the interval density and interval coverage, and an aggregated cluster containing the starting point, the endpoint MAC address, and the terminal device model identifier corresponding to the starting point is output.

5. The method according to claim 4, characterized in that, The selection of the endpoint MAC address based on the density and coverage of the interval includes: Among the candidate MACs scanned before the scan stops, each candidate MAC is used as a candidate termination point; For each candidate termination point, a comprehensive score is calculated based on the density and coverage of the interval corresponding to the candidate termination point. The candidate termination point with the highest overall score is selected as the endpoint MAC address. The calculation method for the comprehensive score is as follows: score=α×density+β×log(distance+1) Where α is the density weight, β is the span weight, density is the interval density corresponding to the candidate termination point, and distance is the integer difference between the candidate termination point and the starting point.

6. The method according to claim 5, characterized in that, The step of using a machine learning classification model to perform quality evaluation on the initial clusters and retaining clusters that meet preset quality conditions includes: For each aggregation cluster in the initial aggregation cluster, a five-dimensional feature vector is extracted; the five-dimensional feature vector includes: the terminal device model identifier corresponding to the aggregation cluster, the number of aggregated MAC addresses in the aggregation cluster, the density of the aggregation cluster, the span of the aggregation cluster, and the comprehensive score of the aggregation cluster; the density is the ratio of the number of aggregated MAC addresses in the aggregation cluster minus one to the integer difference between the endpoint MAC address and the starting MAC address of the aggregation cluster; the span is the integer difference between the endpoint MAC address and the starting MAC address of the aggregation cluster plus one; The five-dimensional feature vector is input into the machine learning classification model, and the machine learning classification model outputs a quality prediction label, which is used to indicate whether the cluster belongs to a high-quality cluster or a low-quality cluster. Clusters whose quality prediction labels indicate high quality are retained, while clusters whose quality prediction labels indicate low quality are removed.

7. The method according to claim 6, characterized in that, The machine learning classification model is a random forest model, and the training process of the random forest model includes: Several cluster records are sampled from the historical aggregation results. The five-dimensional feature vector is extracted from each cluster record, and quality labels are labeled to form a training sample set. The quality labels include high-quality labels and low-quality labels. Using the five-dimensional feature vector as input and the quality label as output, the hyperparameters of the random forest model are searched using random search cross-validation. The hyperparameters include the number of decision trees, the maximum depth, and the minimum number of sample splits. During cross-validation, hyperparameter combinations that meet preset conditions for classification performance metrics on the validation set are selected to obtain the trained random forest model, and the trained random forest model is persistently saved.

8. The method according to claim 7, characterized in that, The process of merging clusters of terminal devices with the same model identifier and adjacent MAC address spaces when their density meets a preset condition includes: The remaining clusters are arranged in ascending order of their starting MAC addresses to form a sequence of clusters to be merged. Traverse the sequence of clusters to be merged, and for two adjacent clusters with the same terminal device model identifier, calculate the new interval density after merging the two adjacent clusters; When the density of the merged new interval is not lower than the preset merging density threshold, the two adjacent clusters are merged into one cluster. The merged cluster is identified by the starting MAC address of the first cluster of the two adjacent clusters, the ending MAC address of the second cluster of the two adjacent clusters, and the model identifier of the terminal device. When the density of the merged new interval is lower than the preset merge density threshold, the two adjacent clusters are retained as independent clusters; The calculation of the new interval density after merging the two adjacent clusters includes: Obtain the number of aggregated MAC addresses in the previous aggregation cluster and the number of aggregated MAC addresses in the next aggregation cluster, and calculate the total number of aggregated MAC addresses after merging as the sum of the two aggregated MAC address numbers; Obtain the integer difference between the endpoint MAC address of the next aggregation cluster and the starting MAC address of the previous aggregation cluster, and calculate the number of MAC addresses covered by the new interval after merging as the integer difference plus one; The ratio of the total number of aggregated MAC addresses after merging to the number of MAC addresses covered by the new merged interval is calculated as the density of the new merged interval.

9. The method according to claim 8, characterized in that, The step of constructing an ordered index based on the starting MAC address of the aggregated cluster in the final aggregated cluster table, performing range matching on the MAC address to be predicted that is not associated with a terminal device model identifier, and predicting the terminal device model identifier corresponding to the MAC address to be predicted includes: Arrange the final aggregated cluster table in ascending order by the starting MAC address; Extract the starting MAC address of each cluster from the final cluster table after sorting to form an ordered array of starting MAC addresses; The endpoint MAC addresses and terminal device model identifiers of each cluster are arranged in the same order as the ordered starting MAC address array to form the endpoint MAC address array and the terminal device model identifier array, respectively. Convert the MAC address to be inferred into an integer value; Perform a binary search on the ordered array of starting MAC addresses to locate the target index, where the target index is the maximum index value that satisfies the condition that the starting MAC address does not exceed the MAC address to be predicted. Read the endpoint MAC address and terminal device model identifier corresponding to the target index; When the MAC address to be predicted is less than or equal to the destination MAC address, the terminal device model identifier of the MAC address to be predicted is predicted as the terminal device model identifier corresponding to the target index; When the MAC address to be predicted is greater than the endpoint MAC address, the MAC address to be predicted is marked as unmatched.

10. A MAC address intelligent aggregation system for profiling home network devices, characterized in that, The system includes: The data fragmentation preprocessing module is used to obtain the MAC address associated with the terminal device model identifier, divide the MAC address into multiple data fragments according to the manufacturer prefix, and sort the MAC addresses in each data fragment by integer value to obtain an ordered MAC address sequence. The parallel clustering module is used to perform multi-constraint greedy clustering on the ordered MAC address sequence in a parallel processing manner to obtain preliminary clusters; wherein, the multi-constraint greedy clustering defines the expansion window based on the adjacent distance constraint, the model purity constraint and the density constraint, and determines the termination point based on the interval density and the interval coverage to form clusters; The intelligent quality assessment module is used to perform quality assessment on the preliminary aggregated clusters using a machine learning classification model, and retain aggregated clusters that meet preset quality conditions; The interval merging module is used to merge clusters of terminal devices with the same model identifier and adjacent MAC address spaces when the density after merging meets the preset conditions, to form the final cluster table. The inference matching module is used to construct an ordered index based on the starting MAC address of the aggregated cluster in the final aggregated cluster table, perform range matching on the MAC address to be inferred that is not associated with the terminal device model identifier, and infer the terminal device model identifier corresponding to the MAC address to be inferred.

Images