A batch identification generation method and system based on large model technology

Abstract

This invention relates to a batch identifier generation method and system based on large-scale model technology, comprising a large-scale model identifier generation module, an identifier batch generation module, and an identifier selection module. In the offline stage, the large-scale model identifier generation module generates an identifier dictionary set based on a vocabulary using a large language model. The identifier batch generation module vectorizes the identifiers using a text embedding model, constructs a similarity matrix for threshold filtering and deduplication, and automatically divides the identifiers into multiple groups using a clustering algorithm, ensuring maximum difference between groups through batch-to-batch difference verification. In the online stage, the identifier selection module dynamically selects the identifier dictionary according to the operating scenario, supporting multiple strategies such as rotation, randomization, and weighting, and includes a built-in real-time conflict detection mechanism. This invention balances the semantic readability and batch uniqueness of identifiers, achieving automated, highly differentiated identifier production and dynamic secure switching. It can be widely applied to scenarios such as financial system interface parameter management, API security protection, and risk management identifier allocation.

Description

Technical Field

[0001] This invention relates to the field of computer software technology, and in particular to a batch identifier generation method and system based on large model technology. Background Technology

[0002] Financial systems typically consist of multiple subsystems or applications (APPs), which may need to be deployed in different countries or specific environments. During deployment, there is a need to switch between interface parameters, languages, and other representations. In particular, frequent switching may be required during interface parameter switching to prevent security threats such as external attacks. Furthermore, in risk management scenarios, different users and different submodules need to be assigned unique identifiers with clear semantics to support accurate risk tracking and control.

[0003] In existing technologies, the following two schemes are commonly used for identifier management: Mapping systems based on manual dictionaries rely on manually setting the mapping dictionary to map the original identifier to the target identifier; however, this approach is slow to update, difficult to cope with frequent switching requirements, has high manual maintenance costs, and the repetition rate is difficult to guarantee, and there may be identifier conflicts between different mapping dictionaries. Mapping systems based on random strings generate strings as identifiers using random algorithms. While this can quickly generate a large number of identifiers, the generated identifiers lack readability and business meaning, which makes subsequent operation, maintenance, and auditing difficult. In addition, there may still be conflicts between randomly generated identifiers, and there is a lack of systematic duplicate detection and conflict elimination mechanisms.

[0004] Therefore, existing identifier generation solutions have the following core problems: manual dictionary methods are slow to update, making it difficult to cope with frequent switching requirements, and the manual maintenance cost is high; although random strings can generate identifiers quickly, they cannot guarantee semantics, and the identifiers lack business meaning; both manual dictionaries and random strings lack systematic duplicate detection and conflict elimination mechanisms, making it difficult to guarantee the uniqueness and difference of identifiers in batch generation scenarios. Summary of the Invention

[0005] In view of this, the present invention provides a batch identifier generation method and system based on large model technology to solve the technical problems of existing identifier generation schemes that cannot balance semantic protection and batch uniqueness and lack a systematic duplicate detection mechanism, and to provide an identifier generation system that supports automated production and dynamic security switching.

[0006] To achieve the above objectives, the present invention provides a batch identifier generation method based on large model technology, comprising the following steps: S1. Generate an offline tag dictionary set based on the vocabulary list, and generate a tag dictionary set based on the large language model; S2. Perform offline batch processing on the set of identifier dictionaries generated by the large language model; S201. Use a text embedding model to convert each identifier in the identifier dictionary set into a high-dimensional vector representation; S202. Calculate the cosine similarity between all pairwise identifier vectors and construct... Similarity matrix ,in Matrix elements Indicator and logo Semantic similarity between them; S203, Set a similarity threshold The similarity matrix is ​​scanned, and when the matrix element... At that time, the judgment mark and logo If the similarity is too high, a greedy strategy is used to remove duplicates, prioritizing the retention of icons with lower average similarity to other icons and removing the other icon, resulting in a set of deduplicated icon vectors. S204. The clustering algorithm is used to automatically divide the deduplicated set of identifier vectors into K groups, where K represents the expected number of identifier dictionaries to be generated. The clustering goal is to make the semantics of identifiers within the same group similar and to maximize the differences between identifiers in different groups. S205. Calculate the batch similarity matrix for the K groups of identifier dictionaries and verify that the average similarity between any two batches is lower than the preset batch difference threshold. If it does not meet the requirement, adjust the clustering parameters and re-batch until the batch difference requirement is met. S3. Based on the operating scenario, select an identifier dictionary online from the identifier dictionary, perform identifier mapping on the input raw data, and generate identifiers.

[0007] Preferably, the vocabulary includes Each word corresponds to a set of words, and generates a corresponding set of words. A total of 1 identifier was generated. The specific steps for generating a dictionary set of identifiers based on a large language model and vocabulary include: S101. Select a prompt word template that matches the current task requirements from a preset prompt word template set through a prompt word selection question. The prompt word template includes at least one configuration parameter in the target language, such as identifier length constraint, semantic similarity requirement, and target language. S102. Input the prompt word template and vocabulary into the large language model, and the large language model generates... A collection of identifier dictionaries for each identifier.

[0008] Preferably, the greedy strategy for deduplication specifically involves: calculating the identifiers separately. and logo Based on the average similarity with all other identifiers, prioritize retaining the identifier. and logo The identifier with the lowest average similarity is removed, and this process is repeated until the similarity of all identifier pairs no longer exceeds a preset similarity threshold. .

[0009] Preferably, the specific steps for dynamically selecting the identifier dictionary online according to the operating scenario include: S301. Receive raw data and currently running scenario parameters, wherein the scenario parameters include at least one of subsystem type, deployment area, and security level; S302. Select the target identifier dictionary from K identifier dictionaries according to the preset selection strategy; S303. Generate an identifier based on the target identifier dictionary for the input raw data; S304. Perform real-time conflict verification to ensure that the currently generated identifier does not duplicate the previously assigned identifier. If a conflict is detected, the identifier reselection will be automatically triggered, and the identifier generated from the original data will be output.

[0010] Preferably, the selection strategy includes a round-robin strategy, a random strategy, and a weighted strategy; The rotation strategy uses each identifier dictionary sequentially in order; The randomization strategy is based on a cryptographically secure random number generator that randomly selects an identifier dictionary. The weighting strategy calculates and selects weights based on the matching degree between scene parameters and each identifier dictionary, and selects weights according to the weight probability distribution.

[0011] This invention also provides a batch identifier generation system based on large model technology, comprising: The large-scale identifier generation module is used to generate an identifier dictionary set based on a vocabulary and a large language model, including a prompt word set, a prompt word selector, and a large language model. The prompt word set contains multiple preset prompt word templates for identifier generation. The prompt word selector is used to select a prompt word template from the prompt word set according to the current task requirements. The large language model is used to generate the identifier dictionary set based on the selected prompt word template and the input vocabulary. The identifier batch generation module is used to perform vectorization, similarity matrix calculation, threshold deduplication, and clustering batch processing on the identifier dictionary set, outputting multiple non-conflicting identifier dictionaries. It includes a vectorization unit, a similarity matrix calculation unit, a threshold filtering and deduplication unit, a clustering batch unit, and an inter-batch verification unit. The vectorization unit uses a text embedding model to convert identifiers into high-dimensional vectors. The similarity matrix calculation unit constructs a global similarity matrix between identifiers. The threshold filtering and deduplication unit performs deduplication based on a greedy strategy. The clustering batch unit uses a clustering algorithm to divide the identifiers into K groups. The inter-batch verification unit verifies whether the differences between each group meet preset requirements. The identifier selection module is used to select one dictionary from multiple representation dictionaries for identifier mapping based on the running scenario during online runtime. It includes a scenario recognizer, a strategy selection engine, a dictionary indexer, a conflict detector, and an identifier cache and version manager. The scenario recognizer is used to identify the current running environment parameters. The strategy selection engine supports multiple dictionary selection strategies. The dictionary indexer establishes an efficient index structure for K identifier dictionaries. The conflict detector performs real-time conflict verification. The identifier cache and version manager maintains multiple version records of the identifier dictionary and supports hot switching.

[0012] Preferably, the dictionary indexer establishes an efficient index structure for K identifier dictionaries, supporting identifier lookup with O(1) time complexity, and the identifier cache and version manager support non-stop updates of the identifier dictionary and historical version rollback.

[0013] Compared with the prior art, the beneficial effects of the present invention are: This invention generates semantically meaningful identifiers through a large language model, and ensures the global uniqueness of the identifiers through matrix-level similarity calculation and threshold deduplication mechanism, thus solving the contradiction between "semantics" and "uniqueness" in traditional solutions. This invention automates the entire identifier generation process, from vocabulary input to multiple identifier dictionaries output, without manual intervention. It can generate an M×N level identifier candidate set at once and automatically batch it, significantly improving identifier production efficiency. In addition, this invention adopts a global deduplication scheme of text embedding vectorization + cosine similarity matrix. Compared with the traditional pairwise string comparison method, it can not only find identifiers with similar surface forms, but also identify implicit repetitions at the semantic level, resulting in higher deduplication accuracy. Furthermore, through the dual mechanism of clustering batching and inter-batch difference verification, it ensures that the differences between K identifier dictionaries are large enough to effectively distinguish them during identifier switching, enhancing the system's security protection capabilities. The prompt word selector of this invention supports multiple identifier generation strategy configurations, and the identifier selection module supports multiple selection strategies (rotation / random / weighted), which can be flexibly adjusted according to different business scenarios. In addition, the identifier selection module has a built-in conflict detection and version management mechanism, supports dictionary hot switching and historical backtracking, and ensures the stability and reliability of the system's online operation. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the batch identifier generation system based on large model technology of the present invention; Figure 2 This is a structural diagram of the large model identifier generation module of the present invention; Figure 3 This is a structural diagram of the batch generation module of the present invention; Figure 4 This is a structural diagram of the identification selection module of the present invention. Detailed Implementation

[0015] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0016] Example 1

[0017] This embodiment provides a batch identifier generation method based on large model technology, including the following steps: S1. Generate an offline tag dictionary set based on the vocabulary list, and generate a tag dictionary set based on the large language model; A vocabulary is a collection of text strings composed of words, symbols, etc. A vocabulary includes... Each word corresponds to a set of words, and generates a corresponding set of words. A total of 1 identifier was generated. The specific steps for generating a dictionary set of identifiers based on a large language model and vocabulary include: S101. Select a prompt word template that matches the current task requirements from a preset prompt word template set through a prompt word selection question. The prompt word template includes at least one configuration parameter in the target language, such as identifier length constraint, semantic similarity requirement, and target language. S102. Input the prompt word template and vocabulary into the large language model, and the large language model generates... A collection of identifier dictionaries for each identifier.

[0018] S2. Perform offline batch processing on the set of identifier dictionaries generated by the large language model; S201. A text embedding model is used to convert each identifier in the identifier dictionary set into a high-dimensional vector representation. The text embedding model encodes the semantic information of the identifier into a fixed-dimensional numerical vector, laying the foundation for subsequent similarity calculation. S202. Calculate the cosine similarity between all pairwise identifier vectors and construct... Similarity matrix , build Similarity matrix That is, the collection of identifier dictionaries includes Each identifier, after being vectorized, yields a vector combination. Matrix elements Indicator and logo Semantic similarity between them, matrix elements The expression is: in, Represents the dot product of vectors. Representing vectors Norm, The range of values ​​is The closer the value is to 1, the more significant the identifier. and logo The more similar the meanings between them; S203, Set a similarity threshold (In this embodiment) The similarity matrix is ​​scanned, and when the matrix element... At that time, the judgment mark and logo If the identifiers are too similar, a greedy strategy is used to remove duplicates, and the identifiers are calculated separately. and logo Based on the average similarity with all other identifiers, prioritize retaining the identifier. and logo The identifier with the lowest average similarity is removed, and this process is repeated until the similarity of all identifier pairs no longer exceeds a preset similarity threshold. Prioritize retaining identifiers with low average similarity to other identifiers and remove the other identifier to obtain a set of deduplicated identifier vectors, thereby maximizing overall dissimilarity; S204. Use a clustering algorithm (K-Means clustering or spectral clustering algorithm) to automatically divide the deduplicated identifier vector set into K groups, where K represents the number of identifier dictionaries to be generated. The clustering goal is to make the identifiers in the same group semantically similar (to maintain semantic consistency in the same application scenario) and maximize the identifier differences between different groups (to facilitate effective differentiation when switching between different scenarios). The number of clusters K is dynamically determined based on the number of subsystems deployed in the system or the frequency of security switching. The objective function of K-Means clustering is to minimize the sum of squares within the group. The goal of clustering is to make the semantics of the identifiers within the same group similar (to facilitate semantic consistency in the same application scenario), while maximizing the difference in identifiers between different groups (to facilitate effective differentiation when switching between different scenarios). In this embodiment, the K-Means++ algorithm is used for cluster center initialization to improve clustering quality. The number of iterations is set to 100, and the convergence condition is that the change in cluster centers is less than 100. ; S205. Calculate the batch similarity matrix for the K groups of identifier dictionaries and verify that the average similarity between any two batches is lower than the preset batch difference threshold. If it does not meet the requirement, adjust the clustering parameters and re-batch until the batch difference requirement is met. The specific verification method is as follows: Calculate the mean cosine similarity between each pair of batches; Check whether all batch pairs meet the condition that the mean cosine similarity is less than the preset inter-batch difference threshold; If any pair of batches does not meet the difference requirement, adjust the clustering parameters and re-batch: Increase the number of clusters K (e.g., K=K+1), replace the K-Means algorithm with the spectral clustering algorithm, or adjust the cluster center initialization parameters and re-execute the clustering; Repeat the above steps until all batches meet the difference requirements; By performing batch-to-batch difference verification, we ensure that the differences between the K identifier dictionaries are large enough to effectively distinguish them during identifier switching, thereby enhancing the system's security capabilities.

[0019] S3. Based on the operating scenario, select an identifier dictionary online from the identifier dictionary, perform identifier mapping on the input raw data, and generate identifiers; S301. Receive raw data and currently running scenario parameters, wherein the scenario parameters include at least one of subsystem type, deployment area, and security level; S302. Select the target identifier dictionary from K identifier dictionaries according to the preset selection strategy; Selection strategies include round-robin, random, and weighted strategies. The rotation strategy uses each identifier dictionary in sequence; if the dictionary currently in use is number i, then the dictionary number to be used next is (i mod K) + 1. This strategy is suitable for scenarios where each dictionary needs to be used evenly to ensure that the usage frequency of each dictionary is balanced. The random strategy is based on a cryptographically secure random number generator to randomly select the identifier dictionary to increase unpredictability. This strategy is suitable for high-security scenarios that require increased unpredictability, preventing attackers from inferring the identifier mapping relationship through the dictionary usage pattern. The weighting strategy calculates and selects weights based on the matching degree between scene parameters and each identifier dictionary, and selects according to a weight probability distribution. The weight of the k-th dictionary is set as follows: The probability of choosing is The factors considered in the weight calculation include: The degree to which the prompt word template used when generating the dictionary matches the security level of the current scene; Historical usage frequency of the dictionary (prioritize dictionaries with lower usage frequency); The degree of matching between the semantic domain of the dictionary and the business domain of the current subsystem; S303. Generate an identifier based on the target identifier dictionary for the input raw data; S304. Perform real-time conflict verification to ensure that the currently generated identifier does not duplicate the previously assigned identifier. If a conflict is detected, the identifier reselection will be automatically triggered, and the identifier generated from the original data will be output.

[0020] Example 2

[0021] like Figure 1 As shown, this embodiment provides a batch identifier generation system based on large model technology, which consists of three core modules: a large model identifier generation module, an identifier batch generation module, and an identifier selection module; The large model identifier generation module is used to generate a set of identifier dictionaries based on a vocabulary and a large language model; for example... Figure 2 As shown, it includes a set of prompt words, a prompt word selector, and a large language model; The prompt word set contains a series of preset prompt word templates for identifier generation, for example: Given a vocabulary list, generate N similar words with different spellings for each word, and output the dictionary set in a specified format; Given a vocabulary, generate N identifiers for each word that have the same semantics but different linguistic expressions, with the target language being {lang}, while maintaining consistency in business semantics; Based on a given vocabulary, generate N semantically related but surface-form differences for each word to obfuscate parameters in high-security scenarios. The prompt word selector is used to select a prompt word template from the prompt word set according to the current task requirements. The selection logic includes: When the target language is Chinese, select a prompt word template that includes Chinese semantic constraints (e.g., template B, lang=zh); When the identifier length constraint is short (e.g., 4-8 characters), select a prompt word template that includes abbreviation generation rules; When the semantic similarity requirement is high (e.g., the generated identifier is required to have a semantic similarity of ≥0.90 with the original vocabulary), select a prompt word template that includes depth control for synonym expansion; When the security level is high, template C is selected, which requires maximizing the difference in surface form between the generated identifier and the original vocabulary. The selected prompt word template and vocabulary list are input into the large language model. Based on its semantic understanding and generation capabilities, the large language model generates N corresponding tags for each word in the vocabulary list. M words generate a total of M×N tags, forming a tag dictionary set. The large-model tag generation module leverages the semantic understanding and generation capabilities of large language models to replace traditional manual dictionary compilation. This significantly improves tag generation speed while ensuring the semantic relevance and readability of the generated tags. Through flexible configuration of the prompt word selector, it can adapt to the differentiated tag requirements of various business scenarios.

[0022] The identifier batch generation module is used to perform vectorization, similarity matrix calculation, threshold deduplication, and clustering batch processing on the identifier dictionary set, outputting multiple non-conflicting identifier dictionaries. This ensures that the identifiers within the same dictionary are semantically cohesive, while maximizing the identifier differences between different dictionaries. Figure 3 As shown, it includes a vectorization unit, a similarity matrix calculation unit, a threshold filtering and deduplication unit, a clustering and batching unit, and an inter-batch verification unit; The vectorization unit uses a text embedding model to convert the identifiers into high-dimensional vectors. The text embedding model converts each identifier in the identifier dictionary set into a high-dimensional vector representation. In this embodiment, a pre-trained BERT model or a Sentence-BERT model is used as the text embedding model to encode the identifier text into a 768-dimensional or 1024-dimensional numerical vector. The text embedding model encodes the semantic information of the identifier into a fixed-dimensional numerical vector, laying the foundation for subsequent similarity calculation. The similarity matrix calculation unit constructs a global similarity matrix between identifiers; The threshold filtering and deduplication unit performs deduplication operations based on a greedy strategy; The clustering batch unit uses a clustering algorithm to divide the identifiers into K groups; The batch-to-batch verification unit verifies whether the differences between groups meet the preset requirements; The identifier selection module is used to select one dictionary from multiple representation dictionaries for identifier mapping based on the runtime scenario, such as... Figure 4As shown, the identifier selection module plays a role in the online operation phase of the system. It is responsible for selecting the most suitable identifier from the pre-generated K identifier dictionaries for identifier mapping based on the input raw data and the current running scenario parameters. The identifier selection module includes a scenario recognizer, a strategy selection engine, a dictionary indexer, a conflict detector, and an identifier cache and version manager. The scene recognizer is used to identify the parameters of the current running environment. The strategy selection engine supports a variety of dictionary selection strategies. The dictionary indexer establishes an efficient index structure for K identifier dictionaries and supports identifier lookup with O(1) time complexity. The conflict detector performs real-time conflict verification. The identifier cache and version manager maintain multi-version records of the identifier dictionary, support hot switching, non-stop update of the identifier dictionary, and historical version backtracking. Dictionary hot-swapping: Load the new version of the identifier dictionary and switch to the new version without stopping system services. The old version dictionary continues to support unfinished identifier mapping requests during the transition period, ensuring zero downtime. Historical version backtracking: Records metadata such as the generation time, version number, and usage scenario of each identifier dictionary, and supports backtracking to historical versions for auditing or troubleshooting when necessary; Cache management: Frequently used identifier dictionaries are cached in memory to reduce disk I / O overhead and improve the response speed of online identifier mapping; In this embodiment, a hash table (HashMap) is used as the index structure, with the original data as the key and the target identifier as the value; for large-scale identifier dictionaries, sharded hash or B+ tree index structures can be used to balance memory usage and query performance. In this embodiment, conflict detection is achieved by maintaining a global hash index of assigned identifiers. When generating a target identifier for some original data, the global hash index is queried first: If the identifier is not assigned, mark it as assigned and output it; If the identifier has already been assigned, the identifier re-selection mechanism is triggered: Select the next available identifier from the same identifier dictionary; If no available identifier is found in the same dictionary, switch to another identifier dictionary for selection; If no available identifiers are found in any dictionary, an alarm will be triggered and the administrator will be notified to expand the dictionary.

[0023] Before the system starts (offline phase), a vocabulary is first constructed, which contains a set of text strings containing all possible words, symbols and so on. The vocabulary is input into the large model identifier generation module, which produces a set of identifier dictionaries. Then, the set of identifier dictionaries is input into the identifier batch generation module, which generates K non-conflicting identifier dictionaries through matrix-level similarity calculation and clustering batching. After the system starts up (online phase), input the raw data and scene parameters, and the identifier selection module selects the most suitable identifier dictionary from K identifier dictionaries to generate and output the target identifier for the raw data; This invention employs a systematic and innovative design based on the TRIZ invention methodology. Through TRIZ contradiction analysis, it is found that traditional identifier generation systems suffer from a technical contradiction between "semantic guarantee" and "batch uniqueness"—ensuring identifiers have clear semantics requires manual intervention or semantic modeling (increasing complexity), while ensuring the uniqueness of a large batch of identifiers requires global conflict detection (increasing computational overhead). To address this contradiction, this invention utilizes multiple TRIZ invention principles for innovation: Structural innovation (TRIZ Principle 1 - Segmentation Principle): The entire identifier generation system is divided into three independent modules: a large model identifier generation module, a machine learning-based batch identifier generation module, and an identifier selection module. Identifier production is completed offline, and only selection is performed online, providing batch and automatic identifier generation capabilities. Mediator Innovation (TRIZ Principle 24 - Mediator Principle): The traditional method of relying on manual construction of the tag dictionary and generation of tags is replaced by a large language model. The large model acts as a "semantic mediator" and automatically transforms the vocabulary into a semantically related tag set, balancing semantic protection and generation efficiency. Matrix-level deduplication innovation (TRIZ principle 17 - dimensional transformation principle): In the batch label generation module, the pairwise comparison problem between labels is transformed into a matrix-level similarity calculation problem. The text embedding model is used to vectorize the labels and construct a similarity matrix. Global deduplication is achieved through threshold filtering to avoid excessively high similarity between multiple labels in the same batch. Clustering Batch Innovation (TRIZ Principle 5 - Merging Principle): After deduplication, the identifiers are automatically grouped into multiple identifier dictionaries through a clustering algorithm. The semantic cohesion of identifiers within the same batch is maximized, and the differences between batches are maximized, thus achieving high-quality batch identifier production. Dynamic Selection Innovation (TRIZ Principle 15 - Dynamic Principle): The identifier selection module dynamically selects the most suitable identifier dictionary based on the runtime scenario (subsystem type, deployment environment, security level, etc.), supports multiple selection strategies such as rotation, random, and weighted selection, and has a built-in real-time conflict detection mechanism.

[0024] Example 3

[0025] This embodiment provides a scenario for the application of the method and system provided by the present invention.

[0026] Scenario 1: Switching the interface parameter identifier of the financial system In the core banking system, multiple subsystems (such as the core accounting system, payment gateway, and risk control system) interact through API interfaces. To prevent external attackers from launching targeted attacks using fixed interface parameters, it is necessary to periodically switch the interface parameter identifiers.

[0027] The system employing this invention first constructs a vocabulary containing all interface parameter names (such as "accountNumber", "transactionAmount", "userId", etc.); the large model identifier generation module generates N=5 semantically related but different identifiers for each parameter (such as "accountNumber" can generate "acctNo", "account_id", "aNum", "accountRef", "acctCode"); the identifier batch generation module divides the generated identifiers into K=3 dictionaries to ensure maximum difference between dictionaries; during the online phase, the strategy selection engine rotates the identifier dictionary every 24 hours according to the current security level and switching frequency requirements, realizing dynamic obfuscation of interface parameters and effectively improving API security.

[0028] Scenario 2: Risk Management Identification Assignment In the risk management system of financial institutions, it is necessary to assign unique risk management identifiers with clear semantics to different users and different business modules in order to support accurate risk tracking and control.

[0029] The system of this invention constructs a vocabulary containing terms such as user type, business module, and risk level. A large-scale identifier generation module generates semantic identifiers (e.g., "retail_loan_high_risk_001"). An identifier batch generation module ensures identifier uniqueness through matrix-level deduplication and groups identifiers of the same risk level into the same dictionary through clustering. In the online phase, a scene recognizer identifies the current business module type, a strategy selection engine selects a matching identifier dictionary, and a conflict detector ensures that the assigned identifiers are globally unique, achieving accurate allocation and tracking of risk identifiers.

[0030] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A batch identification generation method based on large model technology, characterized in that, Includes the following steps: S1. Generate an offline tag dictionary set based on the vocabulary list, and generate a tag dictionary set based on the large language model; S2. Perform offline batch processing on the set of identifier dictionaries generated by the large language model; S201. Use a text embedding model to convert each identifier in the identifier dictionary set into a high-dimensional vector representation; S202、Calculate the cosine similarity between all pairs of identification vectors, and construct a similarity matrix wherein the matrix element represents the semantic similarity between the identification and the identification ; S203, Set a similarity threshold The similarity matrix is ​​scanned, and when the matrix element... At that time, the judgment mark and logo If the similarity is too high, a greedy strategy is used to remove duplicates, prioritizing the retention of icons with lower average similarity to other icons and removing the other icon, resulting in a set of deduplicated icon vectors. S204. The clustering algorithm is used to automatically divide the deduplicated set of identifier vectors into K groups, where K represents the expected number of identifier dictionaries to be generated. The clustering goal is to make the semantics of identifiers within the same group similar and to maximize the differences between identifiers in different groups. S205. Calculate the batch similarity matrix for the K groups of identifier dictionaries and verify that the average similarity between any two batches is lower than the preset batch difference threshold. If it does not meet the requirement, adjust the clustering parameters and re-batch until the batch difference requirement is met. S3. Based on the operating scenario, select an identifier dictionary online from the identifier dictionary, perform identifier mapping on the input raw data, and generate identifiers.

2. The batch identifier generation method based on large model technology according to claim 1, characterized in that, The vocabulary includes Each word corresponds to a set of words, and generates a corresponding set of words. A total of 1 identifier was generated. The specific steps for generating a dictionary set of identifiers based on a large language model and vocabulary include: S101. Select a prompt word template that matches the current task requirements from a preset prompt word template set through a prompt word selection question. The prompt word template includes at least one configuration parameter in the target language, such as identifier length constraint, semantic similarity requirement, and target language. S102. Input the prompt word template and vocabulary into the large language model, and the large language model generates... A collection of identifier dictionaries for each identifier.

3. The batch identifier generation method based on large model technology according to claim 1, characterized in that, The greedy strategy for deduplication specifically involves: calculating the identifiers separately. and logo Based on the average similarity with all other identifiers, prioritize retaining the identifier. and logo The identifier with the lowest average similarity is removed, and this process is repeated until the similarity of all identifier pairs no longer exceeds a preset similarity threshold. .

4. The batch identifier generation method based on large model technology according to claim 1, characterized in that, The specific steps for dynamically selecting the identifier dictionary online based on the operational scenario include: S301. Receive raw data and currently running scenario parameters, wherein the scenario parameters include at least one of subsystem type, deployment area, and security level; S302. Select the target identifier dictionary from K identifier dictionaries according to the preset selection strategy; S303. Generate an identifier based on the target identifier dictionary for the input raw data; S304. Perform real-time conflict verification to ensure that the currently generated identifier does not duplicate the previously assigned identifier. If a conflict is detected, the identifier reselection will be automatically triggered, and the identifier generated from the original data will be output.

5. The batch identifier generation method based on large model technology according to claim 4, characterized in that, The selection strategies include round-robin strategies, random strategies, and weighted strategies; The rotation strategy uses each identifier dictionary sequentially in order; The randomization strategy is based on a cryptographically secure random number generator that randomly selects an identifier dictionary. The weighting strategy calculates and selects weights based on the matching degree between scene parameters and each identifier dictionary, and selects weights according to the weight probability distribution.

6. A batch identifier generation system based on large model technology, employing the method described in any one of claims 1-5, characterized in that, include: The large-scale identifier generation module is used to generate an identifier dictionary set based on a vocabulary and a large language model, including a prompt word set, a prompt word selector, and a large language model. The prompt word set contains multiple preset prompt word templates for identifier generation. The prompt word selector is used to select a prompt word template from the prompt word set according to the current task requirements. The large language model is used to generate the identifier dictionary set based on the selected prompt word template and the input vocabulary. The identifier batch generation module is used to perform vectorization, similarity matrix calculation, threshold deduplication, and clustering batch processing on the identifier dictionary set, outputting multiple non-conflicting identifier dictionaries. It includes a vectorization unit, a similarity matrix calculation unit, a threshold filtering and deduplication unit, a clustering batch unit, and an inter-batch verification unit. The vectorization unit uses a text embedding model to convert identifiers into high-dimensional vectors. The similarity matrix calculation unit constructs a global similarity matrix between identifiers. The threshold filtering and deduplication unit performs deduplication based on a greedy strategy. The clustering batch unit uses a clustering algorithm to divide the identifiers into K groups. The inter-batch verification unit verifies whether the differences between each group meet preset requirements. The identifier selection module is used to select one dictionary from multiple representation dictionaries for identifier mapping based on the running scenario during online runtime. It includes a scenario recognizer, a strategy selection engine, a dictionary indexer, a conflict detector, and an identifier cache and version manager. The scenario recognizer is used to identify the current running environment parameters. The strategy selection engine supports multiple dictionary selection strategies. The dictionary indexer establishes an efficient index structure for K identifier dictionaries. The conflict detector performs real-time conflict verification. The identifier cache and version manager maintains multiple version records of the identifier dictionary and supports hot switching.

7. The batch identifier generation system based on large model technology according to claim 6, characterized in that, The dictionary indexer establishes an efficient index structure for K identifier dictionaries, supporting identifier lookup with O(1) time complexity. The identifier cache and version manager support non-stop updates of the identifier dictionary and historical version rollback.

Images