An ai model product management and service system based on word customization

Abstract

This invention relates to the field of large-scale model multi-tenant service technology, specifically to an AI model artifact management and service system based on lexical customization. This invention achieves independent splitting and storage of custom lexical content and multi-level version verification by setting up a lexical incremental artifact management unit; it provides services through a basic model resident service unit that enables multiple tenants to share the same memory for a single basic model; and it achieves temporary vocabulary splicing, local identifier mapping, isolated inference, and computational resource reclamation within the request lifecycle through a request-level virtual vocabulary construction unit and a request-isolated inference unit. Without modifying the global basic model weights and the global vocabulary, it supports accurate identification of custom lexicals from different tenants, significantly reducing the hardware cost of multi-tenant lexical customization, avoiding cross-tenant lexical conflicts and version compatibility issues, and can be widely applied to multi-tenant lexical customization service scenarios on SaaS-based large-scale model platforms.

Description

Technical Field

[0001] This invention relates to the field of large-scale multi-tenant service technology, and more specifically, to an AI model artifact management and service system based on lexical customization. Background Technology

[0002] With the widespread adoption of generative large models in SaaS service scenarios, many industry tenants have a demand for custom lexical customization. Existing multi-tenant lexical customization solutions mainly fall into two categories. One type trains and deploys an independent customized full model for each tenant, with model instances from different tenants being independent of each other. Under this solution, the size of each model instance can reach tens of gigabytes, and the hardware cost increases linearly with the number of tenants, making it difficult for small and medium-sized tenants to bear the usage costs. Moreover, repeated model loading will cause a large waste of GPU memory resources. The other type of solution merges all tenants' custom lexicals into a global vocabulary, and all tenants share the modified vocabulary and model. This solution will cause the global vocabulary to continuously expand with the number of tenants, and the inference performance will continue to decline. Furthermore, homonyms from different tenants will cause recognition conflicts, making it difficult to meet the tenant's specific semantic recognition needs. At the same time, existing solutions generally lack a sound version compatibility verification mechanism. When the lexical artifacts and the base model versions are incompatible, semantic recognition errors are likely to occur, affecting service stability. To solve this technical problem, we provide an AI model artifact management and service system based on lexical customization. Summary of the Invention

[0003] The purpose of this invention is to provide an AI model artifact management and service system based on lexical customization to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, an AI model artifact management and service system based on lexical customization is provided, including: The lexical incremental artifact management unit is located in the model registration center. It separates the tenant's customized lexical content from the full model and stores it separately as a lexical customized incremental artifact decoupled from the basic model artifact. It performs version compatibility verification between the registered lexical customized incremental artifact and the basic model, and outputs the lexical customized incremental artifact that passes the verification and can be mounted and used. The basic model resident service unit is deployed on the graphics processor. It loads the general basic model instance only once, retains the original vocabulary and original embedding layer of the general basic model, receives the verification results output by the lexical incremental artifact management unit, and only allows the mounting of lexical customized incremental artifacts that have passed the verification, providing shared basic inference capabilities for all tenants. The request-level virtual vocabulary construction unit receives user inference requests carrying tenant identifiers. Based on the tenant identifier, it pulls the corresponding tenant-specific incremental vocabulary from the incremental vocabulary management unit. It temporarily splices the basic vocabulary and the tenant-specific vocabulary only within the processing cycle of the current request to construct a temporary virtual vocabulary that only applies to the current request. It outputs the vocabulary mapping relationship specific to the current request without modifying the global basic vocabulary. The request isolation inference unit performs text segmentation based on the basic word segmenter and the temporary virtual vocabulary of the current request. It maps the tenant's custom words to valid local identifiers within the current request, concatenates the original embedding vector of the basic model with the embedding vectors corresponding to the tenant's words to complete the model's forward computation, and maps the local identifiers back to the corresponding word strings when outputting the results. After the current request is processed, the corresponding temporary virtual vocabulary is destroyed, thus achieving request-level isolation of custom words for different tenants.

[0005] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention reduces the storage and transmission costs of tenant-customized lexical content by splitting it from the full model into independent KB / MB-level incremental lexical artifacts and storing them decoupled from the base model. A single base model resides in the GPU memory, providing shared inference services to all tenants and avoiding memory waste caused by repeated loading of multiple instances. A single GPU can support lexical customization services for hundreds of tenants, reducing hardware costs by two orders of magnitude compared to independent deployment solutions. A request-level temporary virtual vocabulary mechanism dynamically assembles the vocabulary and isolates lexical elements within the lifecycle of a single request, without modifying the global vocabulary or base model weights. This avoids performance degradation caused by global vocabulary expansion and resolves the problem of identifying homonyms among different tenants. A multi-layered version compatibility verification mechanism verifies artifact compatibility from multiple dimensions, including file integrity, lexical identification, and embedding space, preventing semantic errors caused by version mismatches and achieving hard isolation of data between tenants, thus ensuring the security of tenant-customized content. Attached Figure Description

[0006] Figure 1 This is an overall block diagram of the present invention. Detailed Implementation

[0007] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0008] In SaaS-based large-scale model platforms serving multiple enterprise tenants, different industry tenants have numerous needs for using proprietary terms. Financial tenants require models to accurately identify industry-specific terms such as anti-money laundering and non-standard assets; medical tenants need models to accurately identify medical-specific terms such as surgical names and disease abbreviations; and internet tenants need models to identify internal project codes and product jargon. Traditional solutions either deploy customized full-scale models separately for each tenant, leading to a linear increase in hardware costs with the number of tenants, making it difficult to cover the costs for small and medium-sized tenants; or merge all tenant-customized terms into a global vocabulary, causing problems such as homonym recognition errors, infinite vocabulary expansion, and decreased inference performance. To address the challenge of simultaneously satisfying the requirements of terminology customization cost, recognition accuracy, and inference performance in multi-tenant scenarios, this invention provides an AI model artifact management and service system based on terminology customization. Please refer to [link to relevant documentation]. Figure 1 As shown, it includes: The lexical incremental artifact management unit is located in the model registration center. It separates the tenant's customized lexical content from the full model and stores it separately as a lexical customized incremental artifact decoupled from the basic model artifact. It performs version compatibility verification between the registered lexical customized incremental artifact and the basic model, and outputs the lexical customized incremental artifact that passes the verification and can be mounted and used. The basic model resident service unit is deployed on the graphics processor. It loads the general basic model instance only once, retains the original vocabulary and original embedding layer of the general basic model, receives the verification results output by the lexical incremental artifact management unit, and only allows the mounting of lexical customized incremental artifacts that have passed the verification, providing shared basic inference capabilities for all tenants. The request-level virtual vocabulary construction unit receives user inference requests carrying tenant identifiers. Based on the tenant identifier, it pulls the corresponding tenant-specific incremental vocabulary from the incremental vocabulary management unit. It temporarily splices the basic vocabulary and the tenant-specific vocabulary only within the processing cycle of the current request to construct a temporary virtual vocabulary that only applies to the current request. It outputs the vocabulary mapping relationship specific to the current request without modifying the global basic vocabulary. The request isolation inference unit performs text segmentation based on the basic word segmenter and the temporary virtual vocabulary of the current request. It maps the tenant's custom words to valid local identifiers within the current request, concatenates the original embedding vector of the basic model with the embedding vectors corresponding to the tenant's words to complete the model's forward computation, and maps the local identifiers back to the corresponding word strings when outputting the results. After the current request is processed, the corresponding temporary virtual vocabulary is destroyed, thus achieving request-level isolation of custom words for different tenants.

[0009] After the tenant's customized full model file is uploaded, the lexical incremental artifact management unit first obtains the corresponding version of the original base model file. It then divides the full model file and the original base model file into contiguous blocks of a fixed size. The byte similarity of corresponding blocks in the two files is calculated block by block, with a single byte as the smallest unit. The proportion of bytes in the two file blocks that are completely identical to the total number of bytes in the file blocks is calculated, and the calculation process satisfies the following conditions: ,in For the first The byte similarity of each file block, with a dimension of 1 and a value ranging from 0 to 1. For the first The number of bytes in a file block where the contents of two files are completely identical, measured in bytes. This is a fixed number of bytes in a single file block, measured in bytes.

[0010] When the byte similarity of a single file block is 1, the corresponding file block is marked as having the same content. When the byte similarity of a single file block is less than 1, the corresponding file block is marked as having differences. The first and last positions of consecutively marked file blocks having differences together constitute the range of differences. Taking the full-scale custom model file of the lexical uploaded by the financial tenant as an example, the total size of the full-scale model file is 13GB. After being divided into consecutive file blocks of 1MB each, the byte similarity of blocks 1 to 12870 is 1, the byte similarity of blocks 12871 to 12874 is less than 1, and the byte similarity of blocks 12875 to the last block is 1. At this point, the range of differences can be located in the file interval corresponding to blocks 12871 to 12874. Compared with the existing full-file hash comparison, this block comparison method can accurately locate the difference position. It can quickly lock the interval where the incremental content is located without traversing the entire file byte by byte, which greatly reduces the computational overhead of comparing large model files.

[0011] After locating the scope of the discrepancies, the entire model file within this scope is extracted and compared byte by byte with the corresponding content in the original base model file. Bytes that are completely identical to those in the original base model file are removed. The remaining content is the incremental content containing only tenant-defined lexical content. Taking the full model file of the aforementioned financial tenant as an example, the first half of block 12871 in the discrepancy scope is completely identical to the original base model file. After removing the overlapping content, the remaining content consists of three parts: the first part is the set of word embedding vectors corresponding to the new lexicals added by the financial tenant; the second part is the word segmentation matching rules for the new lexicals; and the third part is the configuration parameters after adjusting the vocabulary size. These three parts together constitute the incremental content of the tenant-defined lexical content. The total size of the incremental content is only 2.3MB, which is more than 99.9% smaller than the 13GB full model file, significantly reducing artifact storage costs.

[0012] After extracting the incremental content, the incremental content is encapsulated in an independent format. The encapsulation process adds a fixed-length file header to the incremental content. The file header records the original basic model version identifier corresponding to the generation of the incremental content. After encapsulation, a lexical customized incremental artifact is formed that is completely decoupled from the basic model artifact. The lexical customized incremental artifact can be stored and transmitted independently and can be mounted and used without relying on the full model file.

[0013] After the encapsulation of the lexical-customized incremental artifact is completed, a globally unique artifact identifier is assigned to each lexical-customized incremental artifact. The artifact identifier generation process first extracts the string content of the original basic model version number, concatenates it with the string content of the tenant identifier, and then concatenates it with the first 8 bits of the SHA256 hash value of the lexical-customized incremental artifact content. The three parts are connected by hyphens to form the final globally unique artifact identifier. This identifier generation method can directly identify the basic model version and the tenant to which the lexical-customized incremental artifact belongs through the identifier itself, and can complete the preliminary verification without additional database queries. Compared with the existing general auto-incrementing ID, it can reduce the query overhead of the verification process.

[0014] After generating a globally unique artifact identifier, the globally unique artifact identifier is bound and stored with the corresponding tenant identifier. The binding relationship is recorded in key-value pair form, where the key is the tenant identifier and the value is a list of globally unique artifact identifiers for all lexical customized incremental artifacts of the corresponding tenant. After binding, the lexical customized incremental artifacts are stored in the specified storage path of the model registry center. The storage path contains the globally unique artifact identifier. When retrieving lexical customized incremental artifacts later, the storage path can be located directly through the globally unique artifact identifier without traversing the entire storage directory.

[0015] Once storage is complete, the incremental lexical artifact management unit can trigger the subsequent version compatibility verification process. The entire splitting process does not require modification of the original base model file and the full model file. All comparison and extraction operations are completed in the memory copy, which will not damage the original file content. The customized incremental lexical artifacts obtained by splitting are extremely small, which can significantly reduce artifact storage costs and transmission overhead in multi-tenant scenarios.

[0016] After the lexical custom incremental artifact is registered and stored, the version compatibility verification process is automatically triggered. The lexical incremental artifact management unit first locates the file header of the lexical custom incremental artifact. The file header is a fixed-length header content added during the encapsulation of the lexical custom incremental artifact, which stores the original basic model version identifier corresponding to the incremental generation and the file header verification value. After extracting the file header content, the verification hash value of the file header content is first calculated. The verification hash value calculation performs cyclic redundancy verification on all bytes of the file header on a byte-by-byte basis. The calculation process satisfies the following conditions: ,in The check hash value calculated from the file header is an unsigned integer. This represents all the bytes contained in the file header, measured in bytes.

[0017] The calculated verification hash value is compared with the expected verification value stored in the file header. If the two values ​​are inconsistent, it means that the file header was damaged during storage and transmission. The corresponding token customized incremental artifact is directly determined to not meet the mounting conditions, the corresponding mounting request is intercepted and the verification failure result of file corruption is returned. If the two values ​​are consistent, the original basic model version identifier is read from the fixed offset position of the file header. The original basic model version identifier is a globally unique version string assigned when the basic model is released, which contains three parts: major version number, minor version number, and revision number.

[0018] After extracting the version identifier carried by the lexical customization incremental artifact, a version query request is sent to the basic model resident service unit. After receiving the query request, the basic model resident service unit reads the version identifier of the currently loaded and running basic model, and at the same time reads the end general lexical identifier and embedding layer statistical parameters of the currently loaded basic model, and returns the above information to the lexical incremental artifact management unit.

[0019] After obtaining the version identifiers from the two sources, the string content of the two version identifiers is compared character by character. Taking the lexical customized incremental product uploaded by the financial tenant as an example, the original basic model version identifier stored in the file header of the lexical customized incremental product is v1.2.3, and the basic model version identifier loaded by the current basic model resident service unit is v1.2.3. If the character comparison is completely consistent, the subsequent content conflict verification stage will be entered. If the lexical customized incremental product uploaded by the medical tenant is generated based on the v1.1.5 version basic model and the version identifier stored in the file header is v1.1.5, there is a character difference compared with the currently loaded v1.2.3 version identifier. It is directly determined that the corresponding lexical customized incremental product does not meet the mounting conditions, the corresponding mounting request is intercepted and a version mismatch verification failure result is returned.

[0020] After the version identifiers match, the content conflict verification step begins. First, the starting lexical identifier corresponding to the newly added lexical element is extracted from the lexical customization incremental artifact. The starting lexical identifier is the first lexical identifier assigned to the incremental lexical element during customization. The difference between the extracted starting lexical identifier and the final generic lexical identifier of the currently loaded base model is calculated, and the calculation process satisfies… ,in This is the offset of the lexical identifier, with a dimension of 1. The initial lexical identifier for the incremental artifact is customized for the lexical, with the dimension of an unsigned integer. This is the end-of-line generic term identifier for the currently loaded base model, with dimensions of an unsigned integer.

[0021] Under normal circumstances, the starting lexical identifier of a lexical-customized incremental artifact should be immediately adjacent to the end-of-model general lexical identifier, with an offset value of 1. If the offset value is not 1, it indicates a conflict in lexical identifier allocation, and the corresponding lexical-customized incremental artifact is directly determined not to meet the mounting conditions. The mounting request is then intercepted, and a verification failure result for lexical identifier conflict is returned. After the lexical identifier offset verification passes, the mean and variance statistics of the word embedding vectors in the lexical-customized incremental artifact are further extracted. Cosine similarity is calculated between these values ​​and the mean and variance statistics of the currently loaded base model embedding layer. The cosine similarity calculation process satisfies... ,in To embed statistical similarity, the unit is 1, and the value ranges from 0 to 1. The mean vector of the embedded vectors in the incremental artifact is customized for each word, with the same dimensions as the embedded vectors. This is the mean vector of the current base model's embedding layer, with dimensions consistent with the embedding vector. Customize the standard deviation vector of the embedded vector in the incremental artifact for the lexical, with the same dimensions as the embedded vector. This is the standard deviation vector of the current base model's embedding layer, with dimensions consistent with the embedding vector dimension.

[0022] When the similarity value of the embedding statistics is greater than the preset threshold, it is determined that the embedding space is aligned normally and there is no content conflict. When the similarity value of the embedding statistics is less than or equal to the preset threshold, it indicates that the embedding vector of the lexical customized incremental product is not aligned with the current basic model embedding space. After mounting, semantic understanding errors will occur. It is directly determined that the corresponding lexical customized incremental product does not meet the mounting conditions, the corresponding mounting request is intercepted and the verification failure result of embedding space mismatch is returned.

[0023] After all verification steps are passed, the corresponding custom incremental artifact is marked as mountable and added to the mountable custom incremental artifact list. This list is dynamically updated, updating synchronously each time a mountable artifact is added or removed. The list records the globally unique artifact identifier, corresponding tenant identifier, and storage path for each mountable custom incremental artifact. After the update is complete, the latest mountable custom incremental artifact list is synchronized to the basic model's resident service unit for use in subsequent request processing. This multi-layered, progressive verification mechanism, compared to existing schemes that only verify version numbers, avoids conflicts between version numbers but corrupted actual files and conflicting artifact identifiers. This addresses the implicit problem of embedded space mismatch, preventing incompatible custom lexical artifacts from being mounted onto the base model and causing online service failures. Furthermore, the verification process does not require loading the complete custom lexical artifact content; it only needs to read the file header and a few statistical parameters to complete the verification. The verification process has low resource consumption and will not affect the efficiency of normal request processing. Custom lexical artifacts that fail verification will not be added to the mountable list and will not be synchronized to the base model's permanent service units. When a corresponding tenant initiates a request, the corresponding custom lexical artifact will not be loaded, and the service will automatically revert to using the base vocabulary. This ensures that the incremental artifact issues of a single tenant will not affect the normal use of other tenants.

[0024] When the platform service starts, the basic model resident service unit is deployed in the graphics processor's video memory space. At startup, a general basic model instance is loaded once. The general basic model instance is a general generative large model running instance that has been uniformly trained and verified by the platform. The instance contains a complete model computation graph and model weights that have been loaded into the video memory. It does not contain any tenant-customized content and can independently complete inference calculations for general domain text.

[0025] After the general basic model instance is loaded, the original vocabulary and original embedding layer of the general basic model are retained. The original vocabulary is a fixed set of words determined during the training phase of the basic model. It contains all commonly used words and special tags in the general domain. The mapping relationship between words and word tags is fixed and cannot be modified. The original embedding layer is the embedding weight matrix on the input and output sides of the basic model. The row order of the matrix corresponds one-to-one with the word order of the original vocabulary. The weight vector is fixed after training and provides a general semantic vector representation for all words.

[0026] After completing the weight and vocabulary loading, a service readiness check is performed. This check involves inputting a pre-defined standard test text into the model and comparing the model's output with the pre-defined standard output in terms of character matching rate. The character matching rate is calculated using a single character as the smallest unit, counting the proportion of characters in the output that are completely identical to the standard output out of the total number of characters in the standard output. The calculation process satisfies... ,in The service readiness match rate is a unit of 1, and its value ranges from 0 to 1. The number of characters in the model output that matches the standard output, measured in characters. This represents the total number of characters in standard output, measured in characters.

[0027] When the service readiness matching rate reaches the preset threshold, the basic model is determined to be in a running state that can provide services, and the inference service port is opened to receive requests. When the service readiness matching rate does not reach the preset threshold, the weight loading process is re-executed until the detection passes. This readiness detection mechanism can avoid service anomalies caused by incomplete model loading and ensure that the deployed basic model instance can provide services normally.

[0028] After the basic model resident service unit completes startup and enters service status, it continuously listens for update messages sent by the lexical incremental artifact management unit, receives the verification results output by the lexical incremental artifact management unit, and the updated list of mountable lexical customized incremental artifacts. The list of mountable lexical customized incremental artifacts is structured list data. Each item in the list records a globally unique artifact identifier for a mountable lexical customized incremental artifact, the corresponding tenant identifier, storage path, and number of lexical artifacts. Each time the lexical incremental artifact management unit completes new artifact verification or removes invalid artifacts, it generates a new version of the list of mountable lexical customized incremental artifacts. The list carries a version number and is sent to the basic model resident service unit. After receiving the new list, the basic model resident service unit first compares the list version number. If the version number is the same as the version number of the list currently stored in memory, no processing is performed. If the version number is updated, the list in memory is replaced with the new version list. The replacement process does not interrupt the inference request being processed.

[0029] The resident service unit of the basic model controls the mounting permissions of lexical custom incremental artifacts based on the received verification results and the latest list of mountable lexical custom incremental artifacts. Only lexical custom incremental artifacts that are in the mountable state recorded in the list are allowed to perform mounting operations, and all mounting requests for lexical custom incremental artifacts not in the list are intercepted. The mounting operation does not modify the original weights and original vocabulary of the basic model. The corresponding lexical custom incremental artifact content is dynamically loaded according to the tenant to which the request belongs during the request processing stage. It is not necessary to load the lexical custom incremental artifacts into the video memory in advance.

[0030] Once the tokenized incremental artifacts of both the financial and medical tenants pass the verification and enter the mountable list, both tenants' tokenized incremental artifacts will have mounting permissions. If a tenant's tokenized incremental artifact fails the verification, it will not appear in the mountable list, and the corresponding mounting request will be directly blocked.

[0031] The resident service unit of the basic model keeps the loaded basic model running continuously, providing shared basic inference capabilities for all tenants. During operation, the basic model weights are not switched, unloaded, or reloaded based on different tenants' requests. All tenant requests reuse the same basic model weights in the video memory. Only during the request processing phase are the corresponding tenant's lexical customization incremental content dynamically loaded. After the request processing is completed, the temporary video memory space occupied by the incremental content is immediately released. Compared with the solution of loading an independent model instance for each tenant, this resident operation mechanism can reduce video memory usage by two orders of magnitude. A single graphics processing card can support lexical customization services for hundreds of tenants.

[0032] During the operation of the basic model resident service unit, health checks are continuously performed, and standard test text is periodically run to calculate the service readiness matching rate. If the matching rate is lower than the threshold, a hot reloading process is triggered to reload the weights of the basic model. The loading process adopts a double buffering mechanism, keeping the old instance providing services until the new model instance is loaded, so there will be no service interruption.

[0033] After the basic model resident service unit opens the inference port, the request-level virtual vocabulary construction unit receives the inference request sent by the user. First, it parses the tenant identifier carried in the request header from the fixed offset position. The tenant identifier is a globally unique string assigned by the platform to each registered tenant, used to distinguish the requests and customized content of different tenants. The parsing process verifies the legality of the tenant identifier format. If the tenant identifier format is invalid, it directly returns a request format error response. If the tenant identifier format is valid, it encapsulates the tenant identifier into a query request and sends it to the lexical incremental artifact management unit to query the lexical customized incremental artifact bound to the corresponding tenant identifier.

[0034] After receiving a query request, the lexical incremental artifact management unit queries the storage binding relationship based on the tenant identifier and returns the query result to the request-level virtual vocabulary construction unit. The query result contains the globally unique artifact identifier and mountable status of all lexical customized incremental artifacts bound to the corresponding tenant.

[0035] After receiving the query results, the request-level virtual vocabulary construction unit first determines whether the corresponding tenant has any bound custom lexical artifacts. If no bound custom lexical artifacts exist, it means that the corresponding tenant has not performed any lexical customization, and the basic vocabulary is directly used as the basis for the current requested vocabulary to enter the subsequent temporary vocabulary concatenation process. If bound custom lexical artifacts exist, it further determines whether each bound custom lexical artifact is in a mountable state. If the corresponding custom lexical artifact is not in a mountable state, the basic vocabulary is also used as the basis for the current requested vocabulary to enter the subsequent process. If the corresponding custom lexical artifact is in a mountable state, it first checks whether the corresponding custom lexical artifact exists in the local incremental cache. The local incremental cache is a memory cache space maintained by the request-level virtual vocabulary building unit. It is used to temporarily store frequently accessed lexical custom incremental artifacts, reducing network overhead from cross-node fetching. Cache hit determination is performed by comparing the hash value of the artifact in the cache with the hash value of the artifact recorded in the query results. If the hashes match, it's considered a cache hit, and the corresponding lexical custom incremental artifact is directly read from the local cache. If the hashes don't match or the corresponding artifact doesn't exist in the cache, it's considered a cache miss, and the corresponding tenant-specific lexical custom incremental artifact is fetched from the storage node of the model registry center. The fetching process is executed asynchronously and non-blockingly, adding requests to a waiting queue during fetching to avoid blocking the processing of other tenant requests. The cache hit rate is calculated during the fetching process to dynamically adjust the cache size. The cache hit rate is the ratio of the number of cache-hit requests to the total number of fetch requests per unit time. The calculation process satisfies... ,in This represents the cache hit rate, with a dimension of 1 and a value range from 0 to 1. This represents the number of cache hits per unit of time, measured in "times". This represents the total number of fetch requests per unit time, measured in units of "times". After calculating the cache hit rate, if the cache hit rate is higher than the preset upper limit, the cache space is appropriately increased to accommodate more frequently accessed token-customized incremental artifacts. If the cache hit rate is lower than the preset lower limit, the Least Recently Used (LRU) strategy is used to evict artifacts that have not been accessed for a long time in the cache. Dynamically adjusting the cache space size can maximize cache benefits and reduce fetch latency with limited memory.

[0036] After the fetch is complete, the newly fetched lexical customization incremental artifacts are written to the local incremental cache and enter the subsequent temporary dictionary assembly process. When financial and medical tenants that have completed lexical customization send requests, the query results show that both tenants have lexical customization incremental artifacts in a mountable state. If the corresponding artifact is already in the local cache, it is read directly; otherwise, it is fetched from the storage node. When newly registered tenants that have not done lexical customization send requests, the query results show that no lexical customization incremental artifacts are bound, and the basic dictionary is used directly to enter the subsequent process. If a tenant's lexical customization incremental artifact has just been uploaded and has not yet passed the verification, the query results show that the artifact is not in a mountable state, and the basic dictionary is used to enter the subsequent process. Request processing will not be blocked due to a single artifact not being ready.

[0037] When fetching incremental artifacts for customized lexical units, a fetch timeout is set. If the fetch time exceeds the timeout, the fetch is considered to have failed. After the fetch fails, it will fall back to using the basic vocabulary to enter the subsequent process. When returning the result, a lexical customization degradation flag is carried in the response header to inform the tenant that the current request does not use customized lexical units, so as to avoid unexpected result errors for the tenant. The entire fetch process reduces the frequency of cross-node fetching through a caching mechanism, ensures that only legal and usable incremental artifacts for customized lexical units are loaded through multi-level state judgment, and ensures that the request will not be errored even if the fetch fails through an abnormal fallback mechanism. Compared with the existing solution of loading all tenants' customized lexical content in full, it can significantly reduce memory consumption and fetch latency, while ensuring service availability. After the fetch is completed, the request-level virtual vocabulary building unit records the lexical loading status of the current request and passes the status information to the subsequent vocabulary splicing module along with the request context to enter the temporary vocabulary splicing process.

[0038] After completing the retrieval of the lexical customization incremental artifact or confirming that there is no available lexical customization incremental artifact for the current request, the vocabulary concatenation process begins. First, the basic vocabulary content provided by the resident service unit of the basic model is read. During the reading process, the full data of the basic vocabulary is not copied. Only a read-only memory pointer of the basic vocabulary is saved in the processing context of the current request. The read-only memory pointer is a reference to the memory address of the global basic vocabulary. The content of the global basic vocabulary can be read directly through the pointer, but the global data cannot be modified. The reading process does not make any modifications to the globally stored basic vocabulary. This pointer reference method can significantly reduce memory usage compared to copying the full basic vocabulary, because the basic vocabulary is large and copying the full amount for each request will generate a lot of memory overhead.

[0039] After reading the basic vocabulary, the process proceeds to the corresponding branch based on the lexical loading status of the current request. If no corresponding available custom lexical artifacts are available for the current request, the basic vocabulary is used as the basis for the current request's vocabulary construction process. If a corresponding available custom lexical artifact has been retrieved for the current request, the corresponding tenant-specific lexical content is extracted from the retrieved custom lexical artifacts. The tenant-specific lexical content includes all newly added lexical strings and corresponding matching rules for the tenant. After extraction, the lexical conflict rate between the tenant-specific lexicals and the basic vocabulary is calculated. The lexical conflict rate is the proportion of the number of tenant-specific lexical strings that are completely identical to existing lexical strings in the basic vocabulary to the total number of tenant-specific lexical strings. The calculation process satisfies the following conditions: ,in This represents the word conflict rate, with a dimension of 1 and a value range from 0 to 1. This refers to the number of tenant-specific lexical units that overlap with the basic vocabulary, measured in units. This represents the total number of tenant-specific terms, measured in units.

[0040] After calculating the lexical conflict rate, redundant lexical units that completely overlap with the basic vocabulary in the tenant-specific lexical unit are filtered out. Only newly added lexical units that do not exist in the basic vocabulary are retained to avoid matching conflicts caused by duplicate lexical units. After completing the redundancy filtering, the tenant-specific lexical unit and the basic vocabulary are logically concatenated in the processing context of the current request. The logical concatenation does not modify the existing content of the global basic vocabulary, nor does it copy the full data of the basic vocabulary. It only maintains the reading routes of the two vocabulary in the current request context. When reading lexical units, the content of the tenant-specific lexical unit is read first. Lexical units that do not exist in the tenant-specific lexical unit are then read from the content of the global basic vocabulary through pointers. The concatenation process does not overwrite the original lexical units in the basic vocabulary. When there are homonyms between the tenant-specific lexical unit and the basic vocabulary, the routing rule of prioritizing the reading of the tenant-specific lexical unit ensures that the tenant-customized lexical unit is recognized first.

[0041] The financial tenant's specific terminology includes the abbreviation AML corresponding to anti-money laundering semantics, and the medical tenant's specific terminology also includes the abbreviation AML corresponding to acute myeloid leukemia semantics. Neither terminology overlaps with the general terminology in the basic vocabulary. When concatenating, they are added to the routing table of the corresponding requests. When processing financial tenant requests, AML is identified and matched with anti-money laundering semantics first. When processing medical tenant requests, AML is identified and matched with acute myeloid leukemia semantics first. There will be no semantic confusion, nor will the general semantics of AML in the global basic vocabulary be modified.

[0042] During the concatenation process, the memory usage of the currently requested combined vocabulary is calculated. This memory usage is the sum of the memory occupied by the base vocabulary pointer and the actual memory occupied by the tenant-specific lexicons. The calculation process satisfies... ,in The total memory usage of the combined vocabulary is expressed in bytes. This refers to the memory size occupied by the read-only pointer to the basic vocabulary, measured in bytes. The memory size occupied by tenant-specific terms after filtering out redundancy, measured in bytes.

[0043] When concatenating a vocabulary using pointer references, the memory occupied by the read-only pointer is fixed and does not increase with the size of the base vocabulary. Compared to copying the entire base vocabulary, this reduces memory usage by three orders of magnitude. The vocabulary memory overhead for a single request is only related to the size of the tenant-specific vocabulary. After concatenation, the combined vocabulary content of the current request is obtained. The combined vocabulary exists only in the context of the current request and includes the read pointer of the base vocabulary, the filtered tenant-specific vocabulary, and the corresponding read routing rules. It is not written to global storage or accessed by other requests. After obtaining the combined vocabulary, the subsequent temporary virtual vocabulary construction process begins. The entire concatenation process avoids global data modification and full data copying through read-only pointer references, reduces invalid vocabulary loading through redundant vocabulary filtering, and resolves recognition conflicts of homonyms through priority routing rules. It achieves temporary concatenation of tenant vocabulary and the base vocabulary without modifying the global base vocabulary, ensuring the recognition priority of tenant-customized vocabulary and avoiding the deterioration of inference performance caused by the expansion of the global vocabulary.

[0044] After obtaining the combined vocabulary content of the current request, an independent temporary virtual vocabulary instance is created in the private memory space of the current request. The temporary virtual vocabulary instance is structured mapping data that exists only in the private memory of the current request. It does not generate persistent entity files and is only valid within the processing cycle of the current request. The creation process only operates on the private memory space of the request and does not modify the storage content of the global basic vocabulary.

[0045] After creation, all lexical units in the combined vocabulary are traversed, and the boundaries between the basic vocabulary and the tenant-specific vocabulary in the combined vocabulary are identified. A type marker is added to each lexical unit to clarify whether it belongs to the basic vocabulary or the tenant-defined vocabulary. After marking, a vocabulary mapping relationship specific to the current request is established for the corresponding temporary virtual vocabulary. The vocabulary mapping relationship is a key-value pair structure, where the key is the lexical string and the value is the lexical type, matching priority, and the read address of the corresponding embedding vector. During the establishment of the mapping relationship, the matching priority is calculated for each lexical unit. The matching priority is used to determine the order of lexical unit matching during word segmentation; lexical units with higher values ​​are matched first. The calculation process satisfies... in This represents the priority for word matching, with a unit of 1. It is the basic priority value of the basic word element, with a dimension of 1. The length of the basic word unit is the number of characters. Longer basic words have higher priority and conform to the longest matching rule of word segmentation. The base priority value for tenant-defined terms is 1, and the value is always greater than the highest priority value that all base terms can reach. This ensures that the matching priority of tenant-defined terms is always higher than that of all base terms, and ensures that tenant-specific terms are identified first during the current request processing.

[0046] After priority calculation, all tokens in the mapping relationship are traversed, and the similarity of each pair of token strings is calculated to detect token recognition conflicts. Token recognition conflicts refer to tokens whose strings are completely identical or highly similar but semantically different. The similarity calculation is based on a single character as the smallest unit, and the calculation process satisfies... ,in This represents the string similarity between two terms, with a dimension of 1 and a value ranging from 0 to 1. The minimum edit distance between two word strings, measured in characters. and These are the character lengths of the two lexical units, measured in characters.

[0047] When the similarity between two words is 1, it is determined that there is a homograph recognition conflict. If the conflict exists between two basic words, the original matching rules of the basic vocabulary are retained. If the conflict exists between a basic word and a tenant-defined word, the recognition rules of the tenant-defined word are retained first, covering the matching rules of the basic word at the corresponding position.

[0048] When processing requests from financial tenants, the lexicon AML in the temporary virtual vocabulary is marked as a tenant-defined lexicon, with higher priority than all basic lexicons. During word segmentation and matching, it is prioritized for identification as anti-money laundering semantics. When processing requests from medical tenants, the lexicon AML in the temporary virtual vocabulary is also marked as a tenant-defined lexicon, with higher priority than all basic lexicons. During word segmentation and matching, it is prioritized for identification as acute myeloid leukemia semantics. The temporary virtual vocabularys for the two requests are independent of each other, and there will be no semantic confusion. When processing tenant requests without customized AML lexicons, the temporary virtual vocabulary only contains AML lexicons from the basic vocabulary, and the basic semantics are matched normally. For example, the financial tenant's customized lexicon "flow" corresponds to the semantics of account transaction flow, which has a homomorphic conflict with the semantics of "flow" corresponding to the body of flowing water in the basic vocabulary. After detecting the conflict, the tenant's customized semantics are retained first, and the basic semantics are not matched.

[0049] After the conflict verification is completed, a unique request identifier of the current request is bound to the corresponding temporary virtual vocabulary. The vocabulary mapping relationship and the memory address of the temporary virtual vocabulary are only open to the processing thread of the current request. The processing threads of other requests cannot access the corresponding memory space, ensuring that the temporary virtual vocabulary is only visible to the current request and achieving complete isolation of the vocabulary rules of different tenants.

[0050] After verification, the dictionary mapping relationship specific to the current request is output, and the temporary virtual dictionary is provided for use in subsequent word segmentation. After the request is processed, the corresponding private memory space can be released to destroy the temporary virtual dictionary without additional cleanup operations.

[0051] The entire temporary virtual vocabulary construction process is completed entirely within the requested private memory without modifying any global data. By enforcing priority settings, the priority recognition of tenant-defined terms is guaranteed. Thread-level memory isolation avoids interference between terminology rules across tenants. Compared to the scheme of modifying the global vocabulary, it does not cause global vocabulary expansion, nor does it require maintaining an independent copy of the global vocabulary for each tenant. The additional memory and computational overhead are kept at a low level.

[0052] After obtaining the temporary virtual vocabulary and corresponding vocabulary mapping relationship for the current request, the word segmentation process begins. First, the basic word segmenter is called to perform initial word segmentation on the input text of the current request. The basic word segmenter is a general word segmenter trained with the general basic model. It only performs text segmentation based on the global basic vocabulary and is unaware of tenant-defined words. The initial word segmentation process splits the input text into a continuous sequence of basic words according to the segmentation rules of the basic vocabulary, and obtains the initial word segmentation result. All words in the initial word segmentation result are words that exist in the basic vocabulary and will not include tenant-defined words.

[0053] After initial word segmentation, a secondary matching process is performed on the initial word segmentation results based on the word recognition priority recorded in the word mapping relationship corresponding to the temporary virtual word list. The secondary matching process traverses tenant-specific words in descending order of word priority, with higher-priority words participating in the matching first. A fixed-length sliding window is used to traverse the original text positions corresponding to the initial word segmentation results. The sliding window length is the same as the length of the current tenant-specific word to be matched. The window slides from left to right starting from the beginning of the text. For each character position traversed, the matching score between the text within the window and the current tenant-specific word to be matched is calculated. The matching score is calculated with a single character as the smallest unit, and the proportion of characters identical to the word to be matched within the window is counted relative to the window length. The calculation process satisfies the following conditions: ,in This is the score for matching the window text with custom terms, with a dimension of 1 and a value range from 0 to 1. This refers to the number of characters within the window that are completely identical to the tenant-specific token content to be matched, measured in characters. The length of the sliding window is equal to the character length of the tenant-specific term to be matched, and the unit is characters.

[0054] When the matching score is 1, it is determined that the text within the window completely matches the tenant-specific token. The initial token segmentation result contains text content corresponding to the tenant-specific token. At this point, all basic tokens at the corresponding text position in the initial token segmentation result are located, and consecutive basic tokens at the corresponding position are merged into a custom token unit. After merging, the sliding window skips the corresponding length of text position and continues to slide forward to match the remaining tokens, avoiding repeated matching of the same position. After the sliding window has traversed all text positions, if a certain text segment does not match any tenant-specific tokens, the basic token units output by the basic tokenizer are retained without modification.

[0055] When a financial tenant sends an inference request containing the text "This AML transaction involves duration matching of non-standard assets", the basic word segmenter performs initial word segmentation based on the basic vocabulary. It splits the abbreviation AML into three independent single-character units, "non-standard assets" into four independent single-character units, and "duration matching" into four independent single-character units. The initial unit segmentation result is a continuous sequence of single characters and general words. During the secondary matching process, the longest custom unit is matched first according to priority. When the sliding window passes through the corresponding position of AML, the matching score is 1, and the three single-character units are merged into one custom unit. When the sliding window passes through the corresponding position of "non-standard assets", the matching score is 1, and the four single-character units are merged into one custom unit. When the sliding window passes through the corresponding position of "duration matching", the matching score is 1, and the four single-character units are merged into one custom unit. If no custom unit is matched at other positions, the unit output by the basic word segmenter is retained.

[0056] When a healthcare tenant sends an inference request containing the text "Patients with AML need to take anti-contrast nephropathy drugs after surgery," the basic word segmenter will also split AML into three single-character units. During secondary matching, it will recognize that AML is a disease abbreviation character defined by the healthcare tenant, and merge the corresponding single characters into the corresponding custom word unit, avoiding confusion with the word unit definitions of the financial tenant. After matching is completed, all custom word units and basic word units will be arranged according to the original order of the text, maintaining the original order of the words in the text, to obtain the word segmentation result of the current request. The word segmentation result will then be passed to the subsequent word unit identifier mapping stage.

[0057] The entire secondary matching process does not require modification of the built-in dictionary and model parameters of the basic word segmenter. It only completes the recognition and merging of custom words through post-processing of the initial word segmentation. It does not require separate training or modification of the word segmenter for each tenant, and is compatible with the custom word segmentation requirements of any tenant. At the same time, the matching process strictly follows the word segmentation priority to ensure that the tenant's custom words are not split by the basic word segmentation rules. The recognition accuracy is higher than the solution of directly modifying the global dictionary, and the word segmentation performance will not be reduced due to adding too many custom words to the global dictionary.

[0058] After receiving the word segmentation result of the current request, the word identifier mapping process is entered. First, the maximum global word identifier of the currently loaded basic model is queried. The maximum global word identifier is the identifier number corresponding to the last word in the basic vocabulary. The identifiers of all basic words are less than or equal to this value.

[0059] After the query is completed, the initial value of the local identifier for the current request is calculated. The local identifier is a temporary identifier valid only within the current request and will not be written into the global vocabulary's identifier system. The calculation of the initial value of the local identifier ensures that all local identifiers are outside the scope of the global identifier and will not conflict with the basic lexical identifiers. The calculation process satisfies... ,in This is the starting value for the local identifier of the current request, and its dimension is an unsigned integer. This is the largest global token identifier for the currently loaded base model, measured as an unsigned integer.

[0060] After calculating the initial value of the local identifier, an empty token identifier sequence is initialized, the local identifier allocation count is initialized to 0, and the local identifier association table is initialized to empty. The local identifier association table is used to record the association relationship between the local identifier and the corresponding custom token and the corresponding embedded vector address, and is added to the word mapping relationship of the current request.

[0061] After preparation, all word units are traversed in the original order of the word segmentation results. During the traversal, each word unit is distinguished as either a basic word unit or a tenant-defined word unit based on its word unit type tag. For word units belonging to basic words, the corresponding original word unit identifier is directly retrieved from the basic word table, and the retrieved identifier is appended to the end of the word unit identifier sequence without any modification. For word units belonging to tenant-defined words, local identifiers are assigned to them according to the traversal order. The allocation of local identifiers is recursively performed, and the calculation process satisfies the following conditions: ,in For the traversed number of Each custom lexical unit corresponds to a local identifier, with the dimension being an unsigned integer. This is the sequence number of the custom tokens that have been traversed in the current request, starting from 0 and measured in units of "pieces".

[0062] For each local identifier allocated, it is checked whether the corresponding identifier already exists in the local identifier association table. The identifier duplication rate is calculated, which is the proportion of the number of duplicate identifiers among the allocated identifiers to the total number of allocated identifiers. When the identifier duplication rate is 0, it is determined that there is no identifier conflict. The local identifier is appended to the end of the token identifier sequence. At the same time, the association relationship between the local identifier and the corresponding custom token and the corresponding embedded vector address is recorded in the local identifier association table. The local identifier allocation count is incremented by 1. If the identifier duplication rate is not 0, the corresponding local identifier is recalculated and allocated to avoid identifier conflicts within the same request.

[0063] The local identifier allocation process does not modify the identifier content of the global basic vocabulary. All local identifiers are only recorded in the private mapping table of the current request and are not synchronized to the global storage. The maximum global token identifier of the currently loaded basic model is 32000. The identifier range of all basic tokens is 0 to 32000. The starting value of the local identifier requested by the financial tenant is 32001. In the word segmentation result, the first custom token AML is assigned the local identifier 32001, the second custom token non-standard asset is assigned the local identifier 32002, and the third custom token duration matching is assigned the local identifier 32003. The remaining basic tokens use the original identifiers in the range of 0 to 32000. There is no overlap or conflict among the identifiers. The starting value of the local identifier requested by the medical tenant is also 32001. In the word segmentation result, the custom word AML is assigned the local identifier 32001. Although the local identifier value is the same as that of AML in the financial tenant's request, the two identifiers belong to the private mapping tables of different requests and are only valid within the corresponding request. There will be no cross-request identifier confusion.

[0064] After all lexical units have been traversed, the order of the identifiers in the lexical identifier sequence is completely consistent with the order of the lexical units in the word segmentation result, with no disorder. At this point, the lexical identifier sequence of the current request is obtained. The word table mapping relationship, which is supplemented with the local identifier association, is passed together with the lexical identifier sequence to the subsequent forward calculation stage of the model. The entire identifier mapping process does not require modification of the global word table identifier system, does not require allocation of globally unique identifiers for tenant-customized lexical units, and local identifiers of different requests can be reused. There will be no problem of exhausting global identifier resources, nor will the lexical mapping of the basic model be disordered due to the modification of the global identifier correspondence by adding new custom lexical units. The identifier allocation process is simple to calculate, with extremely low additional computational overhead, and will not increase the request processing latency.

[0065] After receiving the lexical identifier sequence and the vocabulary mapping relationship with the local identifier associations for the current request, the model forward calculation and result generation process begins. First, an embedding vector sequence of the corresponding length is initialized according to the length of the lexical identifier sequence, and continuous memory space is reserved to store the embedding vectors.

[0066] Iterate through each lexical identifier in the lexical identifier sequence. First, determine the identifier type, distinguishing between an original identifier belonging to a basic lexical unit and a local identifier belonging to a custom lexical unit. The type determination is directly completed through the identifier's numerical range, without needing to query an additional mapping table. The determination process satisfies... in This is a marker for the identifier type, with a dimension of 1. A value of 1 indicates that the identifier is a local identifier of a custom lexical, while a value of 0 indicates that the identifier is the original identifier of a basic lexical. ID is the identifier of the lexical to be judged, with a dimension of an unsigned integer. This is the largest global lexical identifier of the currently loaded base model, measured as an unsigned integer. If it's determined to be an existing identifier of the base lexical, the embedding vector at the corresponding position is directly read from the original embedding layer of the base model and copied to the corresponding position in the embedding vector sequence. If it's determined to be a local identifier of a custom lexical, the embedding vector of the corresponding custom lexical is read based on the embedding vector address recorded in the currently requested lexical mapping relationship. After reading, the distribution offset distance between the custom embedding vector and the base embedding layer vector is calculated, and the embedding value is verified to meet the model input requirements. The offset distance calculation satisfies... ,in To define the vector offset distance between the custom embedding mean and the base embedding mean, with the same dimensions as the embedding vector. This is the mean vector of all custom token embedding vectors requested in the current request. It is the mean vector of all word embedding vectors in the basic embedding layer.

[0067] When the offset distance is within the preset allowable range, the embedding value is determined to be normal, and the corresponding embedding vector is copied to the corresponding position in the embedding vector sequence. When the offset distance exceeds the preset allowable range, the embedding value is determined to be abnormal, and the embedding vector corresponding to the unknown word in the basic vocabulary is used to replace the embedding vector of the corresponding custom word to avoid abnormal values ​​being input into the model and causing output errors.

[0068] After all the embedding vectors are copied, the embedding vector sequence is a continuous block of memory. It can be directly input into the model backbone network to perform forward computation without additional memory copying. The forward computation generates predicted labels word by word according to the model's autoregressive generation logic until the end label is generated or the maximum generation length is reached, thus obtaining the predicted label sequence output by the model. After obtaining the predicted identifier sequence, each output identifier is traversed. The same numerical range judgment method is used to distinguish whether the identifier belongs to the original identifier of the basic word unit or the local identifier. If it is determined to be the original identifier of the basic word unit, the corresponding word unit string is directly read from the basic word unit table. If it is determined to be the local identifier, the corresponding custom word unit string is read from the local identifier association table of the word unit mapping relationship. The reading process is executed in the order of the predicted identifiers. The read word unit strings are concatenated in order to obtain the final output text. The output text is returned to the corresponding tenant. In the word unit identifier sequence requested by the financial tenant, the local identifiers with values ​​greater than 32000 correspond to three custom words: AML, non-standard assets, and duration matching. After reading the corresponding embedding vector and verifying that it is normal, it is input into the model. In the predicted identifiers output by the model, the local identifiers correspond to the read custom word unit strings. The concatenation results containing financial terms are obtained. In the word unit identifier sequence requested by the medical tenant, the local identifiers with values ​​greater than 32000 correspond to medical terms such as AML. The corresponding embedding and word unit strings are read to generate the correct results for the medical scenario. The embedding reading and forward calculation processes of the two requests are independent of each other and there will be no word unit confusion.

[0069] Once the current request result is returned, all private memory occupied by the current request is immediately released, the temporary virtual vocabulary created for the current request is deleted, all local identifier association records in the vocabulary mapping relationship are deleted, the embedded vector sequence and all temporary computation data are deleted, the released memory is returned to the system memory pool, and all tenant-defined vocabulary data and temporary data generated during request processing are completely cleared and will not remain in memory to be accessed by other requests.

[0070] The entire forward computation and decoding process determines the identifier type by numerical range, which can quickly distinguish between basic lexical units and custom lexical units without additional table lookups. The processing speed is basically the same as the inference speed of the native model. The embedded distribution verification avoids output errors caused by abnormal custom embeddings. The thorough memory reclamation after the request is processed achieves hard isolation of custom lexical units of different tenants. There will be no cross-tenant lexical leakage or data interference, and no memory leakage due to long-term accumulation of temporary data. While supporting multi-tenant custom lexical units, it maintains the original inference performance and stability of the basic model.

[0071] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A management and service system for AI model artifacts based on lexical customization, characterized in that, include: The lexical incremental artifact management unit is located in the model registration center. It separates the tenant's customized lexical content from the full model and stores it separately as a lexical customized incremental artifact decoupled from the basic model artifact. It performs version compatibility verification between the registered lexical customized incremental artifact and the basic model, and outputs the lexical customized incremental artifact that passes the verification and can be mounted and used. The basic model resident service unit is deployed on the graphics processor. It loads the general basic model instance only once, retains the original vocabulary and original embedding layer of the general basic model, receives the verification results output by the lexical incremental artifact management unit, and only allows the mounting of lexical customized incremental artifacts that have passed the verification, providing shared basic inference capabilities for all tenants. The request-level virtual vocabulary construction unit receives user inference requests carrying tenant identifiers. Based on the tenant identifier, it pulls the corresponding tenant-specific incremental vocabulary from the incremental vocabulary management unit. It temporarily splices the basic vocabulary and the tenant-specific vocabulary only within the processing cycle of the current request to construct a temporary virtual vocabulary that only applies to the current request. It outputs the vocabulary mapping relationship specific to the current request without modifying the global basic vocabulary. The request isolation inference unit performs text segmentation based on the basic word segmenter and the temporary virtual vocabulary of the current request. It maps the tenant's custom words to valid local identifiers within the current request, concatenates the original embedding vector of the basic model with the embedding vectors corresponding to the tenant's words to complete the model's forward computation, and maps the local identifiers back to the corresponding word strings when outputting the results. After the current request is processed, the corresponding temporary virtual vocabulary is destroyed, thus achieving request-level isolation of custom words for different tenants.

2. The AI ​​model artifact management and service system based on lexical customization according to claim 1, characterized in that: The specific steps for separating the tenant-defined lexical content from the full model are as follows: The lexical incremental artifact management unit obtains the full model file after tenant lexical customization is completed, as well as the original basic model file that has not undergone lexical customization corresponding to the full model; Among them, the full model file after the tenant completes the lexical customization is a complete model package that can be run independently after the tenant completes the exclusive lexical addition and embedding alignment training based on the general basic model provided by the platform. Its file directory structure and running dependencies are completely consistent with the general basic model without customization. The only customization differences are in the word list configuration and embedding layer weights. The rest of the model structure and weight parameters are consistent with the general basic model. Taking a specific application scenario as an example, when a financial tenant customizes lexical units on the platform, it first uploads a list of exclusive lexical units required for its own business scenario, including exclusive terms for anti-money laundering abbreviations, business product names, and risk control terminology. After receiving the lexical unit list, the platform appends the above-mentioned custom lexical units to the end of the original vocabulary of the general basic model, adjusts the dimensions of the model's input embedding layer and output prediction layer accordingly, and uses the corresponding domain business corpus provided by the tenant to perform incremental training on the embedding vectors of the newly added lexical units. The embedding vectors of the newly added lexical units are aligned to the general semantic space of the basic model. After training is completed, the full model file of the financial tenant's customized lexical units can be exported. When medical tenants customize lexical units, they upload medical-specific lexical units such as disease abbreviations, surgical names, and drug names. Among them, abbreviations that are identical to those of financial tenants correspond to completely different semantics in different tenant scenarios. After the platform completes lexical unit addition, dimension adjustment, and embedding alignment training according to the same process, it exports the full model file of lexical unit customization for medical tenants. From the perspective of file composition, the full model file after lexical customization consists of four parts: model weight file, word segmenter configuration file, word list file, and model runtime configuration file. Among them, the vast majority of the content in the model weight file is completely consistent with the weights of the general base model. Only the vector parts of the corresponding newly added words in the input embedding layer and output prediction layer are customized new content. In the word segmenter configuration file and word list file, the matching rules and ID mapping of general words are completely consistent with the base model. Only the matching rules of tenant-customized words are appended to the end of the word list. In the model runtime configuration file, only the word list size parameter differs from the base model. The other network structure parameters and inference runtime parameters are completely consistent with the general base model. It can be directly loaded into the inference framework and run independently. Because this type of full model file has a large amount of content overlap with the base model and only contains a small amount of tenant-customized incremental content, the incremental content related to custom words can be separated from the full model through subsequent file comparison steps to form independent lexical customized incremental artifacts without the need to store and deploy the complete full model. The original base model file without lexical customization is a general base model package that is uniformly trained by the platform and provides shared inference services to all tenants. It is the unified basis for all tenants to carry out lexical customization. The file structure is fixed and the content has been uniformly verified for security and effect by the platform. It does not contain any tenant-specific customized content. All tenants modify the same version of the original base model file when carrying out lexical customization. Taking the current version of the platform as an example, the original basic model is a 7B parameter generative large model trained on a massive general domain corpus. It has basic capabilities for semantic understanding, content generation, and logical reasoning in general scenarios. The original basic model file contains four fixed components. The first part is the model backbone weight file, which contains the weights of all Transformer network layers except for the input embedding layer and the output prediction layer. It carries all the general semantic understanding and generation capabilities of the model. In the full model after all tenants have completed the lexical customization, the content of this part of the file is completely consistent with the original basic model. The second part is the basic embedding weight file, which contains all the weight vectors of the general vocabulary corresponding to the input embedding layer and the output prediction layer. The order of the vectors is the same as that in the general vocabulary. The word order corresponds one-to-one. The third part is the general word segmenter file, which contains the merging rules of the BPE word segmentation algorithm, text preprocessing logic, and matching rules for all words in the general vocabulary. It can independently complete the word segmentation processing of general text. The fourth part is the basic running configuration file, which contains model network structure parameters, vocabulary size parameters, inference batch processing parameters, and attention mechanism parameters. It provides all the configuration basis for model loading and inference operation. The original basic model file is only loaded once when the basic model resident service unit starts. After loading, its disk storage content and video memory loaded data are set to read-only state. It will not be modified by any tenant's word customization operation or request processing operation. It resides in the graphics processor's video memory for a long time to provide shared basic inference capabilities for all tenants. Compare the contents of the full model file and the original base model file segment by segment to identify the consistent and different parts, and locate the range of differences in the full model file relative to the original base model file. The redundant parts that overlap with the original basic model content are removed from the range of differentiated content, and the incremental content containing only tenant-customized word elements is extracted. The incremental content is then encapsulated in an independent format to form a lexical-customized incremental artifact that is completely decoupled from the basic model artifact. Finally, a globally unique artifact identifier is assigned to the lexical-customized incremental artifact, which is then bound to the identifier of the corresponding tenant. The bound lexical-customized incremental artifact is then stored in the model registry center.

3. The AI ​​model artifact management and service system based on lexical customization according to claim 2, characterized in that: The specific steps for verifying the version compatibility of the registered lexical-customized incremental artifacts with the base model are as follows: After the lexical incremental artifact is registered and stored, the lexical incremental artifact management unit extracts the version identifier of the original basic model corresponding to the lexical customized incremental artifact when it was split and generated. Extract the two version identifiers of the basic model loaded and running in the current basic model resident service unit, and compare the consistency of the two version identifiers; If the two version identifiers are inconsistent, it is directly determined that the custom incremental product of the word element does not meet the mounting conditions, the corresponding mounting request is intercepted and the verification failure result is returned; If the two versions have the same identifier, further verify whether there is a content conflict between the content of the custom incremental artifact and the currently loaded base model. If the verification finds a content conflict, it is determined that the custom incremental product of the word element does not meet the mounting conditions, the corresponding mounting request is intercepted and the verification failure result is returned. If no content conflict is found during verification, the custom incremental artifact for the lexical element is marked as mountable, added to the list of mountable custom incremental artifacts for lexical elements, and the information of the mountable custom incremental artifacts for lexical elements that have passed verification is output to the basic model resident service unit.

4. The AI ​​model artifact management and service system based on lexical customization according to claim 3, characterized in that: The specific operation process of the basic model's resident service unit is as follows: The basic model resident service unit is deployed on the graphics processor. At startup, a general basic model instance is loaded once. After the general basic model instance is loaded, the original vocabulary and original embedding layer of the general basic model are retained to keep the basic model in a running state that can provide services. After the basic model resident service unit is started, it continuously receives the verification results and the updated list of mountable custom incremental artifacts from the lexical artifact management unit. Based on the received verification results, the basic model resident service unit only allows custom incremental artifacts in the mountable state in the updated list of mountable custom incremental artifacts to perform the mounting operation, and intercepts the mounting requests of custom incremental artifacts that fail the verification. The basic model resident service unit keeps the loaded basic model running continuously, providing shared basic inference capabilities for all tenants, and does not repeatedly load the basic model with the requests of different tenants during the operation.

5. The AI ​​model artifact management and service system based on lexical customization according to claim 4, characterized in that: The specific steps for retrieving the corresponding tenant-specific lexical incremental artifact from the lexical incremental artifact management unit based on the tenant identifier are as follows: After receiving a user's inference request, the request-level virtual vocabulary construction unit parses the tenant identifier carried in the inference request and sends the tenant identifier to the lexical incremental artifact management unit. It then queries the lexical customized incremental artifact bound to the tenant identifier. After receiving the query result returned by the lexical incremental artifact management unit, it determines whether the corresponding tenant has a bound lexical customized incremental artifact. If no bound lexical customized incremental artifact exists, the basic vocabulary is used as the basis for the current request's vocabulary and the process proceeds to the next step. If a bound lexical customized incremental artifact exists, it further determines whether the lexical customized incremental artifact is in a mountable state. If the corresponding lexical customized incremental artifact is not in a mountable state, the basic vocabulary is again used as the basis for the current request's vocabulary and the process proceeds to the next step. If the corresponding lexical customized incremental artifact is in a mountable state, the lexical customized incremental artifact specific to the corresponding tenant is retrieved from the lexical incremental artifact management unit. After retrieval, the process proceeds to the subsequent temporary vocabulary splicing process.

6. The AI ​​model artifact management and service system based on lexical customization according to claim 5, characterized in that: The specific steps for temporarily assembling the basic vocabulary and tenant-specific terms are as follows: After completing the retrieval of the lexical customization incremental artifact or confirming that there is no available lexical customization incremental artifact for the current request, the basic vocabulary content provided by the resident service unit of the basic model is read. The reading process does not modify the globally stored basic vocabulary. If there is no corresponding available lexical customization incremental artifact for the current request, the basic vocabulary is directly used as the vocabulary basis for the current request to enter the subsequent temporary virtual vocabulary construction process. If the current request has retrieved the corresponding available lexical customization incremental artifact, the corresponding tenant-specific lexical content is extracted from the retrieved lexical customization incremental artifact. Within the processing context of the current request, the tenant-specific lexical is logically concatenated with the basic vocabulary. The concatenation process does not modify the existing content of the global basic vocabulary, nor does it overwrite the original lexical in the basic vocabulary. After concatenation, the combined vocabulary content of the current request is obtained, and the subsequent temporary virtual vocabulary construction process is entered.

7. The AI ​​model artifact management and service system based on lexical customization according to claim 6, characterized in that: The specific steps for constructing a temporary virtual vocabulary that only applies to the current request are as follows: After obtaining the combined vocabulary content of the current request, an independent temporary virtual vocabulary instance is created for the current request. This temporary virtual vocabulary is only valid within the processing cycle of the current request. The creation process does not modify the storage content of the global basic vocabulary. The boundary between the basic vocabulary part and the tenant-specific vocabulary part of the combined vocabulary is clarified, and it is determined whether each vocabulary belongs to the basic vocabulary or the tenant-defined vocabulary. A vocabulary mapping relationship specific to the current request is established for the temporary virtual vocabulary. The vocabulary mapping relationship records the recognition priority of each vocabulary to ensure that the tenant-specific vocabulary is recognized first during the processing of the current request. After establishing the mapping relationship, it is checked whether there is a vocabulary recognition conflict in the temporary virtual vocabulary. If there is a vocabulary recognition conflict, the recognition rules of the tenant-specific vocabulary are retained first. After the verification is completed, the vocabulary mapping relationship specific to the current request is output, and the temporary virtual vocabulary is provided for use in the subsequent inference stage. The temporary virtual vocabulary is only visible to the current request, and other requests cannot access the content of the temporary virtual vocabulary.

8. The AI ​​model artifact management and service system based on lexical customization according to claim 7, characterized in that: The specific steps for completing text segmentation based on the basic word segmenter combined with the temporary virtual vocabulary of the current request are as follows: After obtaining the temporary virtual vocabulary and corresponding vocabulary mapping relationship for the current request, the basic word segmenter is called to perform initial word segmentation processing on the input text of the current request, and the initial word segmentation result is obtained. The word segmentation result is then matched a second time with the word recognition priority recorded in the vocabulary mapping relationship corresponding to the temporary virtual vocabulary. During the second matching process, tenant-specific words are identified first. If the text content corresponding to the tenant-specific word is identified in the initial word segmentation result, the segmentation result at the corresponding position is merged into a custom word unit. If the text content corresponding to the tenant-specific word is not identified, the basic word unit output by the basic word segmenter is retained. After the matching is completed, all custom word units and basic word units are arranged in the original order of the text to obtain the word segmentation result of the current request.

9. The AI ​​model artifact management and service system based on lexical customization according to claim 8, characterized in that: The specific steps for mapping tenant-defined terms to valid local identifiers within the current request are as follows: After receiving the word segmentation result of the current request, it iterates through all word units in the segmentation result, distinguishing whether each word unit belongs to a basic word unit or a tenant-defined word unit. For word units belonging to basic words, the original word unit identifier corresponding to the basic vocabulary is used directly without modification. For word units belonging to tenant-defined words, a local identifier is assigned to them within the processing context of the current request, which is only valid within the processing cycle of the current request. The local identifier does not modify the identifier content of the global basic vocabulary. After the assignment is completed, the association relationship between the local identifier and the corresponding custom word unit is added to the vocabulary mapping relationship of the current request. After the identifier mapping of all word units is completed, all word identifiers are arranged according to the original order of the word segmentation result to obtain the word identifier sequence of the current request.

10. The AI ​​model artifact management and service system based on lexical customization according to claim 9, characterized in that: The concatenation of the basic model's original embedding vector and the embedding vector corresponding to the tenant's word element completes the model's forward computation. When outputting the result, the local identifier is mapped back to the corresponding word element string. The specific steps for destroying the corresponding temporary virtual vocabulary after the current request is processed are as follows: Upon receiving the token identifier sequence for the current request, each token identifier in the sequence is traversed to distinguish whether it belongs to the original identifier of the basic token or the local identifier of the custom token. For the original identifier of the basic token, the corresponding embedding vector is directly read from the original embedding layer of the basic model and input into the backbone network of the model. For the local identifier of the custom token, the embedding vector of the corresponding custom token is read according to the word list mapping relationship of the current request. All embedding vectors are input into the backbone network of the model in token order to complete the forward computation and obtain the predicted identifier sequence output by the model. After obtaining the predicted identifier sequence, each output identifier is traversed to distinguish whether it belongs to the original identifier of the basic token or the local identifier. For the original identifier of the basic token, the corresponding token string is directly read from the basic word list. For the local identifier, the corresponding custom token string is read according to the word list mapping relationship. The strings are concatenated in order to obtain the final output text. After the current request result is returned, the temporary virtual word list, word list mapping relationship and associated records of the local identifier created for the current request are immediately destroyed, and the relevant resources occupied by the current request are released to achieve request-level isolation of custom tokens of different tenants.

Images