Semantic quantification model training method and semantic identifier generation method and system

Abstract

The embodiment of the invention provides a semantic quantification model training method and a semantic identifier generation method and system. A semantic quantification model comprises M quantification layers which are connected in sequence. And the training system obtains a sample data set, and performs iterative training on each quantization layer in sequence according to the connection sequence of the quantization layers. Each round of iteration of the mth quantization layer comprises the steps that the training system transmits description information of sample objects to the mth quantization layer in sequence, an input vector of each sample is obtained, the coding network of the mth quantization layer is used for coding the input vectors, and a coding vector of each sample is obtained. And then the training system generates each codeword vector in the codebook and the mth hierarchy class target signature associated with the codeword vector based on the hierarchy class target signature and the clustering condition of the coding vector of each sample to obtain a target codeword vector corresponding to each sample so as to determine the semantic alignment loss of the mth quantization layer. And finally, updating the parameters of the coding network at least based on the semantic alignment loss.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a training method for a semantic quantization model, a method for generating semantic identifiers, and a system thereof. Background Technology

[0002] With the rapid development of digital services and e-commerce, recommendation systems or retrieval systems have become a key bridge connecting users with services or goods. In order for recommendation systems or retrieval systems to efficiently handle massive demands, it is usually necessary to transform the descriptive information of an object (such as text or image descriptions of a product) into identifiers that computers can quickly retrieve.

[0003] However, current identifiers output by models often lack sufficient representational power, making it difficult to adequately represent specific objects. This deficiency leads to situations where systems using these identifiers (such as recommendation or retrieval systems) are unable to effectively locate the desired objects, thus affecting the accuracy of their outputs and the user experience. Therefore, training semantic quantization models to output identifiers with strong semantic representation capabilities has become a pressing issue.

[0004] The information in the background section is merely information known only to the inventor and does not imply that such information had entered the public domain before the date of this application, nor does it imply that it can be considered prior art in this disclosure. Summary of the Invention

[0005] This specification provides a training method for a semantic quantization model, a method for generating semantic identifiers, and a system. It can adopt a layer-by-layer iterative training strategy, use multi-level class target tags to guide the generation of codebooks for each quantization layer, and update the parameters of each layer of the encoding network by constraining the degree of difference between the encoded vector output by the quantization layer and the target codeword vector to be aligned, thereby improving the convergence speed of the model and enabling the trained semantic quantization model to output identifier sequences with semantic representation and discriminative power.

[0006] Firstly, this specification provides a training method for a semantic quantization model. The semantic quantization model includes M sequentially connected quantization layers, where M is an integer greater than 1. The method includes: obtaining a sample dataset, where each sample in the sample dataset includes: descriptive information of a sample object and its corresponding multi-level class target tags; and iteratively training each quantization layer sequentially using the sample dataset according to the connection order of the M quantization layers, wherein at least one m-th quantization layer exists among the M quantization layers, where m is an integer less than M. Each iteration of the m-th quantization layer includes: for each sample in the sample dataset, sequentially passing the descriptive information of the sample object from the first quantization layer to the m-th quantization layer, and obtaining the sample. The input vector of the m-th quantization layer is used to encode the input vector of each sample using the encoding network of the m-th quantization layer, resulting in the encoding vector of each sample. Based on the target tag of the m-th level class to which each sample belongs and the clustering of the encoding vectors of each sample, codeword vectors in the codebook corresponding to the m-th quantization layer are generated, and the target tag of the m-th level class associated with each codeword vector is determined. Based on the codeword vectors in the codebook corresponding to the m-th quantization layer and their associated target tags of the m-th level class, the target codeword vectors to be aligned for the encoding vectors of each sample are determined. Based on the target codeword vectors and encoding vectors corresponding to each sample, the semantic alignment loss of the m-th quantization layer is determined, and the parameters of the encoding network in the m-th quantization layer are updated based on at least the semantic alignment loss.

[0007] In some embodiments, the number of samples in the sample dataset is N, and for the i-th sample, where i ranges from 1 to N, the encoding vector of the i-th sample is denoted as . The target codeword vector corresponding to the i-th sample is denoted as . The semantic alignment loss is at least based on and The similarity between them is determined and used to constrain the feature space. Towards near.

[0008] In some embodiments, the codebook, excluding the target codeword vector The codeword vectors other than are denoted as The semantic alignment loss is also based on and The similarity between them is determined and used to constrain the feature space. keep away .

[0009] In some embodiments, the semantic alignment loss It is determined based on the following formula: ,in, Based on codeword vectors The associated m-th level class target signature and codeword vector The weight coefficients determined by the associated m-th level class target tag. This represents the number of codeword vectors in the codebook corresponding to the m-th quantization layer.

[0010] In some embodiments, Determined based on the following formula: ,in, Codeword vector The semantic vector of the associated m-th level class target tag; Codeword vector The semantic vector of the associated m-th level class target tag; express and Semantic relevance between them This is a hyperparameter.

[0011] In some embodiments, each iteration of the m-th quantization layer further includes: for each sample, performing semantic quantization on the encoding vector of the sample from the 1st to the m-th quantization layers to output m codeword vectors, and decoding the m codeword vectors to obtain a reconstructed vector; determining a reconstruction loss based on the difference between the embedding vector corresponding to the description information of the sample object and the reconstructed vector; and updating the parameters of the encoding network in the m-th quantization layer based at least on the semantic alignment loss, including: constructing a target loss based on the reconstruction loss and the semantic alignment loss; and updating the parameters of the encoding network in the m-th quantization layer based on the target loss.

[0012] In some embodiments, each iteration of the m-th quantization layer further includes: determining the layer normalization loss of the m-th quantization layer, wherein the layer normalization loss characterizes the degree to which each codeword vector in the codebook is used evenly; and constructing a target loss based on the reconstruction loss and the semantic alignment loss includes: constructing the target loss based on the reconstruction loss, the semantic alignment loss, and the layer normalization loss.

[0013] In some embodiments, the layer normalization loss It is determined based on the following formula: ,in, The frequency of use of the k-th codeword vector in the codebook. This is a hyperparameter.

[0014] In some embodiments, the M quantization layers further include an nth quantization layer, where n is an integer less than or equal to M. Each iteration of the nth quantization layer includes: for each sample in the sample dataset, passing the description information of the sample object sequentially from the 1st quantization layer to the nth quantization layer, and obtaining the input vector of the sample in the nth quantization layer; encoding the input vector of each sample using the encoding network of the nth quantization layer to obtain the encoding vector of each sample; generating each codeword vector in the codebook corresponding to the nth quantization layer based on the clustering of the encoding vectors of each sample; quantizing the encoding vector of each sample and outputting the codeword vector based on the codeword vectors in the codebook corresponding to the nth quantization layer; determining the layer normalization loss of the nth quantization layer based on the semantic quantization of each sample, wherein the layer normalization loss characterizes the degree to which each codeword vector in the codebook is used evenly; and updating the parameters of the encoding network in the nth quantization layer based at least on the layer normalization loss.

[0015] In some embodiments, the layer normalization loss It is determined based on the following formula: ,in, The frequency of use of the k-th codeword vector in the codebook. This is a hyperparameter.

[0016] In some embodiments, the nth quantization layer is located after the mth quantization layer.

[0017] In some embodiments, when m=1, the input vector is the embedding vector corresponding to the description information of the sample object; when m>1, the input vector is the residual vector between the encoding vector of the (m-1)th quantization layer and the codeword vector output by the (m-1)th layer.

[0018] In some embodiments, based on the m-th level class target tag to which each sample belongs and the clustering of the encoding vectors of each sample, each codeword vector in the codebook corresponding to the m-th quantization layer is generated, and the m-th level class target tag associated with each codeword vector is determined, including: clustering the encoding vectors of each sample to obtain K clusters, and determining the cluster center vectors corresponding to each of the K clusters; using the cluster center vectors corresponding to each of the K clusters as codeword vectors in the codebook corresponding to the m-th quantization layer; and for each codeword vector, determining the m-th level class target tag associated with the codeword vector based on the m-th level class target tag to which each sample in the cluster corresponding to the codeword vector belongs.

[0019] In some embodiments, based on each codeword vector in the codebook corresponding to the m-th quantization layer and its associated m-th level class target tag, the target codeword vector to be aligned for each sample's encoding vector is determined, including: for each sample, based on the m-th level class target tag to which the sample belongs, determining at least one candidate codeword vector associated with the corresponding class target tag from the codebook corresponding to the m-th quantization layer; and taking the candidate codeword vector with the highest similarity to the encoding vector of the sample among the at least one candidate codeword vector as the target codeword vector.

[0020] In some embodiments, the sample object is a service or a product; the description information of the sample object includes description information in at least one modality of text, image, or voice.

[0021] Secondly, this specification also provides a method for generating semantic identifiers. The method includes: obtaining descriptive information of a target object; obtaining a trained semantic quantization model, the semantic quantization model including M sequentially connected quantization layers, each quantization layer including an encoding network and a codebook, the codebook including multiple codeword vectors, where M is an integer greater than 1; inputting the descriptive information of the target object into the semantic quantization model, and performing layer-by-layer quantization and residual propagation through the M quantization layers to obtain codeword identifiers output by each quantization layer; wherein, for any m-th quantization layer, the quantization process includes: encoding the input vector using the encoding network of the m-th quantization layer to obtain an encoding vector; semantically quantizing the encoding vector into a codeword vector in the codebook based on the codebook of the m-th quantization layer, and outputting a codeword identifier for identifying the corresponding codeword vector; and generating an identifier sequence for identifying the target object based on the codeword identifiers output by the M quantization layers.

[0022] Thirdly, this specification also provides a computing system, comprising: at least one storage medium storing at least one instruction set; and at least one processor communicatively connected to the at least one storage medium, wherein, when the computing system is running, the at least one processor reads the at least one instruction set and executes the training method as described in any one of the first aspects, or executes the method as described in the second aspect, according to the instructions of the at least one instruction set.

[0023] The training methods for the semantic quantization model, the methods for generating semantic identifiers, and other functions of the system provided in this specification will be partially listed in the following description. The inventive aspects of the training methods for the semantic quantization model, the methods for generating semantic identifiers, and the system provided in this specification can be fully explained through practice or by using the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram illustrating an application scenario of a training method for a semantic quantization model provided according to an embodiment of this specification is shown. Figure 2 A flowchart is shown of a method for generating semantic identifiers according to embodiments of this specification; Figure 3 A hardware structure diagram of a computing system provided according to an embodiment of this specification is shown; Figure 4 A flowchart illustrating the processing of a semantic quantization model provided according to an embodiment of this specification is shown. Figure 5 A flowchart is shown of a training method for a semantic quantization model provided according to an embodiment of this specification; Figure 6 A schematic diagram of an iterative training of a quantization layer according to an embodiment of this specification is shown; Figure 7 A schematic diagram illustrating a method for determining reconstruction loss according to an embodiment of this specification is shown; and Figure 8 A schematic diagram illustrating the training process of a semantic quantization model provided according to an embodiment of this specification is shown. Detailed Implementation

[0026] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0027] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0028] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0029] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0030] The following section introduces the application scenarios of this manual.

[0031] With the rapid development of technologies such as generative recommendation and semantic retrieval, the key to improving system performance lies in how to transform the complex descriptive information of massive amounts of services or goods (objects, sample objects or target objects in the following text) into identifiers that computers can process efficiently.

[0032] In some embodiments, based on the object's descriptive information, methods based on the Residual Quantization (RQ) framework can be used to generate identifiers or identifier sequences to identify the object, such as Residual-Quantized Variational Autoencoder (RQ-VAE) or Residual-Quantized K-means Clustering (RQ-Kmeans). Both models convert the descriptive information into a high-dimensional vector, input the high-dimensional vector into multiple quantization layers for quantization, and extract the best-matching codeword.

[0033] However, the above solutions have significant limitations. They employ a single encoding network to accommodate the feature extraction needs of all subsequent quantization layers. A single encoding network struggles to extract features suitable for all quantization levels simultaneously, resulting in insufficient feature representation capabilities for deep quantization. Furthermore, because these solutions utilize end-to-end joint training, the parameters of multiple quantization layers are heavily coupled, making convergence difficult with large datasets or deep networks, and the models are sensitive to hyperparameter selection. Secondly, convergence is also difficult with large datasets, and as the number of quantization layers increases, the feature information contained in the deep quantization vectors gradually diminishes, leading to a lack of discriminative power in the generated identifiers (codewords), making it difficult to accurately reflect object features. Moreover, because the codewords matched by each quantization layer lack explicit semantic information, the generated identifier sequences have weak semantic relevance to the application scenario, failing to effectively support retrieval and recommendation systems.

[0034] Therefore, the above-mentioned methods are insufficient to meet current requirements in terms of representation ability, convergence, and discriminative power. This specification provides a training method for a semantic quantization model. It configures an independent encoding network for each quantization layer and iteratively trains the encoding network corresponding to each quantization layer, using multi-level class target tags of objects to guide the generation of codebooks and the semantic quantization process for each quantization layer. This allows each quantization layer to focus on extracting semantic features at a specific level, reducing the problems of deep semantic decay and training convergence difficulties. The trained semantic quantization model can then output identifier sequences with strong semantic representation ability and high discriminative power.

[0035] Figure 1 A schematic diagram illustrating an application scenario for training a semantic quantization model according to an embodiment of this specification is shown. For example... Figure 1 As shown, the application scenario 100 may include: a training system 110 for semantic quantization models (hereinafter referred to as the training system) and a semantic identifier generation system 120 (hereinafter referred to as the generation system).

[0036] See Figure 1 Application scenario 100 can involve two stages: the training stage and the application stage.

[0037] During the training phase, a semantic quantization model is trained based on the training system 110. The semantic quantization model includes M sequentially connected quantization layers, where M is an integer greater than 1.

[0038] The training system 110 first obtains a sample dataset, where each sample includes descriptive information of the sample object (such as text descriptions, images, etc. of a product) and its corresponding multi-level class labels. Then, the training system 110 iteratively trains each quantization layer sequentially using the sample dataset, according to the connection order of the M quantization layers. At least one m-th quantization layer exists among the M quantization layers, where m is an integer less than M. Each iteration of the m-th quantization layer includes the following steps.

[0039] For each sample in the sample dataset, the training system 110 sequentially passes the description information of the sample object from the first quantization layer to the m-th quantization layer, and obtains the input vector of the sample in the m-th quantization layer. Then, the training system 110 uses the independent encoding network of the m-th quantization layer to encode the input vector of each sample, obtaining the encoding vector of each sample. Based on the m-th level class target tag to which each sample belongs and the clustering of the encoding vectors of each sample, the system generates the codeword vectors in the codebook corresponding to the m-th quantization layer, and determines the m-th level class target tag associated with each codeword vector.

[0040] Furthermore, the training system 110 determines the target codeword vector to be aligned for each sample based on each codeword vector in the codebook corresponding to the m-th quantization layer and its associated m-th level class target tag, and determines the semantic alignment loss of the m-th quantization layer based on the target codeword vector and the encoding vector corresponding to each sample.

[0041] The training system 110 updates the parameters of the encoding network in the m-th quantization layer based at least on the semantic alignment loss. The trained semantic quantization model is able to map the descriptive information of an object into a sequence of identifiers with semantic representations.

[0042] In some embodiments, the training system 110 may store data and instructions for implementing the training method of the semantic quantization model provided in this specification, and may execute or be used to execute the data and instructions. In some embodiments, the training system 110 may include a hardware device with data information processing capabilities and the necessary programs required to drive the hardware device to work.

[0043] In traditional unsupervised training methods such as RQ-VAE, training the model requires training the codeword distribution in the feature space from scratch, resulting in a lengthy training process and slow convergence. This specification's embodiments decouple the training of each quantization layer through layer-by-layer iteration and independent encoder design. This addresses the severe semantic decay caused by a single encoding network's inability to adapt to multi-level features, allowing the quantization network to focus on capturing the semantic information of that layer. By generating and determining the target codeword vector based on class target signatures, explicit supervision information is provided to the model. This supervision not only enables the encoding vector to converge to the target codeword vector to be aligned more quickly, improving the training convergence speed, but also avoids the semantic drift problem that easily occurs in unsupervised clustering. Furthermore, based on the aforementioned semantic alignment loss, the identifier sequence generated by the trained model is no longer composed of meaningless random symbols, but rather consists of identifiers carrying semantic information, enabling the identifier sequence to reflect the differences in category affiliation between different objects. The semantic quantization model trained in the above way can generate identifier sequences with semantic representation and discriminative power. This improves the ability of systems that use this identifier (such as retrieval or recommendation systems) to distinguish different objects and the accuracy of retrieval recall based on this identifier.

[0044] During the application phase, the generation system 120 has a semantic quantization model pre-deployed. When it is necessary to generate a semantic identifier corresponding to a target object, the generation system 120 can obtain the descriptive information of the target object and input the descriptive information into the semantic quantization model to generate an identifier sequence for identifying the target object.

[0045] In some embodiments, the specific form of the identifier can be flexibly set according to the encoding requirements of the actual application scenario. For example, the identifier can be identified by numbers (e.g., "1", "2"); the identifier can also be identified by letters (e.g., "A", "B"); the identifier can also be identified by a combination of numbers and letters (e.g., "A1", "B20"), where the letter part can be used to identify the codebook of the quantization layer to which the identifier belongs (e.g., "A" represents the codebook A of the first quantization layer, and "B" represents the codebook B of the second quantization layer), while the number part is used to identify the specific codeword in the codebook of that layer. The identifier sequence composed of the identifiers of each quantization layer can express the quantization results of the sample object in different quantization layers, constructing an identifier with semantic representation. Through the above identifiers, the descriptive information of the sample object is mapped into a short and semantically informative identifier sequence.

[0046] In some embodiments, the generation system 120 may store data and instructions for implementing the semantic identifier generation method provided in this specification, and may execute or be used to execute the data and instructions. In some embodiments, the generation system 120 may include a hardware device with data information processing capabilities and the necessary programs required to drive the hardware device to work.

[0047] It is understood that the training system 110 and the generation system 120 may correspond to the same computing system or different systems, and this specification does not impose any restrictions on this.

[0048] It should be noted that the training system 110 can correspond to a single device or a cluster of devices; this specification does not impose any restrictions on this. When the training system 110 corresponds to a single device, the training method for the semantic quantization model can be executed entirely on that device. When the training system 110 corresponds to a cluster of devices, the training method for the semantic quantization model can be executed collaboratively on multiple devices corresponding to the cluster; this specification does not impose any restrictions on this either.

[0049] The generation system 120 can correspond to a single device or a cluster of devices; this specification does not impose any restrictions on this. When the generation system 120 corresponds to a single device, the semantic identifier generation method can be executed entirely on that device. When the generation system 120 corresponds to a cluster of devices, the semantic identifier generation method can be executed collaboratively on multiple devices corresponding to the cluster; this specification does not impose any restrictions on this.

[0050] It should be noted that all user data obtained in this manual has been authorized by the user and does not involve user privacy.

[0051] Figure 2 A hardware structure diagram of a computing system 200 provided according to an embodiment of this specification is shown. The computing system 200 can be used as... Figure 1 The training system 110 in the specification executes the training method for the semantic quantization model described herein. The computing system 200 can also be used as... Figure 1 The generation system 120 in the specification executes the semantic identifier generation method described herein.

[0052] like Figure 2 As shown, the computing system 200 may include at least one storage medium 230 and at least one processor 220. In some embodiments, the computing system 200 may also include a communication port 250 and an internal communication bus 210. The computing system 200 may also include I / O components 260.

[0053] The internal communication bus 210 can connect to different system components. For example, the internal communication bus 210 can connect to storage medium 230, processor 220, communication port 250, and I / O component 260, etc.

[0054] I / O component 260 supports input / output between computing system 200 and other components.

[0055] Communication port 250 is used for data communication between computing system 200 and the outside world. For example, communication port 250 can be used for data communication between computing system 200 and network 140. Communication port 250 can be a wired communication port or a wireless communication port.

[0056] Storage medium 230 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 232, a read-only storage medium (ROM) 234, or a random access storage medium (RAM) 235. Storage medium 230 also includes at least one instruction set stored in the data storage device. The instruction set may include computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc.

[0057] At least one processor 220 may be communicatively connected to at least one storage medium 230. When the computing system 200 is running, at least one processor 220 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the training method for the semantic quantization model provided in this specification. The processor 220 may execute the steps included in the training method for the semantic quantization model. The processor 220 may be in the form of one or more processors. In some embodiments, the processor 220 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof.

[0058] For illustrative purposes only, the accompanying drawings show only one processor 220 for the computing system 200. However, it should be noted that the computing system 200 may also include multiple processors; therefore, the operations and / or method steps disclosed herein may be executed by one processor or by multiple processors in combination. For example, if the processor 220 of the computing system 200 described in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 220 (e.g., a first processor executes step A, a second processor executes step B, or the first and second processors jointly execute steps A and B).

[0059] Figure 3 A flowchart of a semantic identifier generation method P300 according to an embodiment of this specification is shown. As previously described, the generation system 120 can execute the semantic identifier generation method P300 of this specification. Specifically, the processor in the generation system 120 can read an instruction set stored in its local storage medium and then execute the semantic identifier generation method P300 of this specification according to the provisions of the instruction set. Figure 3 As shown, method P400 may include steps S310-S340.

[0060] S310: The generation system first obtains the description information of the target object.

[0061] The descriptive information of the target object can include multiple modalities, such as at least one modality of text, image, or voice. For example, for products in an e-commerce scenario, the descriptive information may include the product title text, a list of attribute parameters, and the main product image; for services in a lifestyle service scenario, the descriptive information may include a service details introduction and related voice instructions.

[0062] S320: The generation system obtains a trained semantic quantization model, which includes M sequentially connected quantization layers. Each quantization layer includes an encoding network and a codebook, wherein the codebook includes multiple codeword vectors, and M is an integer greater than 1.

[0063] Please see Figure 4 The semantic quantization model is pre-trained based on the training method described in the embodiments of this specification, and each quantization layer includes an encoding network and a codebook.

[0064] S330: Input the description information of the target object into the semantic quantization model, and perform layer-by-layer quantization and residual propagation through the M quantization layers to obtain the codeword identifier output by each quantization layer. For any m-th quantization layer, the quantization process includes: encoding the input vector using the encoding network of the m-th quantization layer to obtain an encoding vector, semantically quantizing the encoding vector into a codeword vector in the codebook based on the codebook of the m-th quantization layer, and outputting a codeword identifier to identify the corresponding codeword vector.

[0065] In some embodiments, the encoding network can be a multilayer perceptron (MLP) or a Transformer encoder layer, configured to extract features and transform the dimensions of the input vector, outputting a fixed-dimensional encoded vector. To reduce computational complexity and adapt to the codebook's storage structure, the dimension of this encoded vector can be set to a low dimension (e.g., 256 dimensions). Through the processing of the encoding network, the input vector can be mapped to a feature subspace suitable for quantization at the current layer. Furthermore, the encoding network can fuse multimodal features, such as aligning text and image features before input, thereby more comprehensively representing the semantic information of the target object. The trained semantic quantization model has multimodal information processing capabilities.

[0066] Please continue reading Figure 4 Regarding the definition of the input vector, there are two cases: the first layer and subsequent layers. In some embodiments, when m=1 (i.e., for the first quantization layer), the input vector is the embedding vector corresponding to the description information of the target object. When m>1, the input vector of the quantization layer is the residual vector passed by the (m-1)th quantization layer, that is, the residual vector between the encoding vector of the (m-1)th quantization layer and the codeword vector output by the (m-1)th layer.

[0067] In some embodiments, the input information into the semantic quantization model can be the original description information of the target object or a pre-processed embedding vector. In practical applications, the generation system can use existing pre-trained models (such as BERT, ResNet, or CLIP models) to extract features from the multimodal description information of the target object, obtain a high-dimensional embedding vector, and use this embedding vector as the input to the first quantization layer.

[0068] In some embodiments, layer-by-layer quantization refers to the process by which each of the M quantization layers maps the vector input to the current quantization layer into an encoded vector through an encoding network, and semantically quantizes the encoded vector into an identifier. The quantization process includes: calculating the similarity (e.g., cosine similarity or Euclidean distance) between the encoded vector of the current quantization layer and each codeword vector in the codebook associated with that layer; selecting the codeword vector with the highest similarity (or closest distance) from the codebook; and using the index of that codeword vector in the codebook as the output identifier.

[0069] In some embodiments, residual propagation refers to each of the M quantization layers calculating the difference between the encoded vector obtained by the current quantization layer's encoding network encoding the input vector and the codeword vector (i.e., the output codeword vector) obtained by the current quantization layer's semantic quantization, and propagating this difference as a residual vector to the next quantization layer, until all M quantization layers have been processed.

[0070] S340: The generation system generates an identifier sequence for identifying the target object based on the codeword identifiers output by the M quantization layers respectively.

[0071] To more intuitively illustrate the semantic quantization process for obtaining each quantization layer, Figure 4 Taking the semantic quantization model, which includes four sequentially connected quantization layers, as an example, and assuming the description information of the target object is "model A IV drip", the processing procedure of each quantization layer is as follows.

[0072] The first quantization layer receives the embedding vector obtained by converting "model A IV drip" as the input vector. This embedding vector is then input into the encoding network of the first quantization layer to obtain the first encoded vector. The first quantization layer calculates the similarity between this first encoded vector and the codeword vectors of each codeword in codebook A, determining the codeword vector with the highest similarity. The codeword A2 corresponding to this codeword vector is used as the identifier output by the first quantization layer. Subsequently, the difference between the first encoded vector and the codeword vector A2 is calculated to obtain the first residual vector.

[0073] The second quantization layer receives the first residual vector as its input vector. This first residual vector is then input into the encoding network of the second quantization layer to obtain the second encoded vector. The second quantization layer calculates the similarity between this second encoded vector and each codeword vector in codebook B, determining the codeword vector with the highest similarity. The codeword B4 corresponding to this codeword vector is used as the identifier output by the second quantization layer. Next, the difference between the second encoded vector and the B4 codeword vector is calculated to obtain the second residual vector.

[0074] The third quantization layer receives the second residual vector as its input vector. This second residual vector is then input into the encoding network of the third quantization layer to obtain the third encoded vector. The third quantization layer, also based on the similarity matching principle, determines the codeword vector C1 with the highest similarity in the codebook C. C1 serves as the identifier output by the third quantization layer, and the difference between the third encoded vector and the C1 codeword vector is further calculated to obtain the third residual vector.

[0075] The fourth quantization layer receives the third residual vector as its input vector and passes it through the encoding network to obtain the fourth encoded vector. In the fourth quantization layer, the codeword vector D2 with the highest similarity to the fourth encoded vector is determined from the codebook D. D2 serves as the identifier output by the fourth quantization layer. The identifiers output by each quantization layer are arranged according to the connection order of the quantization layers, resulting in the identifier sequence (e.g., A2, B4, C1, D2) output by the trained semantic quantization model for the target object's descriptive information. The first three identifiers (A2, B4, C1) are obtained after semantic quantization of the quantization layers trained using the semantic alignment loss described later, representing the semantic information at the levels of "medical," "medical device," and "IV drip," respectively. The fourth identifier (D2) is obtained after semantic quantization of the quantization layers trained using the layer normalization loss described later; its corresponding codeword vector represents the subtle features remaining after stripping away the macro-level semantics of the object (e.g., specific model, specifications, or texture differences). Therefore, based on this identifier sequence, the semantic information of the target object can be represented from macro-level categories (medical, medical device, IV drip) to micro-level details (e.g., a specific model in model AZ). The trained semantic quantization model can compress and map descriptive information into a short identifier sequence. This identifier sequence has semantic representation and discriminative power, improving processing efficiency in subsequent use.

[0076] In order for the semantic quantization model to perform the above semantic identifier generation process, the semantic quantization model needs to be trained in advance.

[0077] Figure 5 A flowchart of a training method P400 for a semantic quantization model according to an embodiment of this specification is shown. As previously illustrated, the training system 110 can execute the training method P400 for the semantic quantization model of this specification, wherein the semantic quantization model comprises M sequentially connected quantization layers, where M is an integer greater than 1. Specifically, the processor in the training system 110 can read the instruction set stored in its local storage medium and then execute the training method P400 for the semantic quantization model of this specification according to the specifications of the instruction set. Figure 5 As shown, method P400 may include steps S410-S420.

[0078] S410: The training system obtains a sample dataset, wherein each sample in the sample dataset includes: description information of the sample object and its corresponding multi-level class target label.

[0079] In some embodiments, the sample object is a service or a product, and the description information of the sample object includes description information in at least one modality of text, image, or voice.

[0080] The sample objects can be services (e.g., transportation services, utility payment services) or goods (e.g., clothing, medical supplies, electronic products). The descriptive information of the sample objects can include multiple modalities. For example, the descriptive information of goods can include the product title, attribute text, and / or the main image of the product. The descriptive information of services can include a textual description of the service details, and / or related audio instructions, etc.

[0081] A multi-level category tag for a sample object refers to a hierarchical tagging system constructed for the sample object to describe its category information at different levels. In some embodiments, the multi-level category tag can be pre-built by a service system providing services or goods. For example, in a transportation service scenario, if the description information for a sample object is "subway QR code," its corresponding multi-level category tag could be: the first-level tag is "transportation," and the second-level tag is "subway travel." As another example, in a medical scenario, if the description information for a sample object is "model A IV drip," its first-level tag could be "medical," the second-level tag is "medical device," and the third-level tag is "IV drip." By introducing multi-level category tags, hierarchical supervisory information can be provided for the training of semantic quantization models.

[0082] S420: The training system iteratively trains each quantization layer sequentially using the sample dataset according to the connection order of the M quantization layers. Among the M quantization layers, there is at least the m-th quantization layer, where m is an integer less than M. Each iteration of the m-th quantization layer includes steps S421-S424.

[0083] S421: For each sample in the sample dataset, the description information of the sample object is sequentially passed from the first quantization layer to the m-th quantization layer, and the input vector of the sample in the m-th quantization layer is obtained.

[0084] In some embodiments, when m=1, the input vector is the embedding vector corresponding to the description information of the sample object; when m>1, the input vector is the residual vector between the encoding vector of the (m-1)th quantization layer and the codeword vector output by the (m-1)th layer.

[0085] In some embodiments, each codebook may contain 256 codewords, and each codeword may be a fixed-dimensional vector (e.g., 256-dimensional). The number of codewords and the dimension of the codewords can be set according to the scale and semantic complexity of the object data in the actual application scenario.

[0086] S422: The training system uses the encoding network of the m-th quantization layer to encode the input vector of each sample to obtain the encoding vector of each sample. Based on the m-th level class target tag to which each sample belongs and the clustering of the encoding vector of each sample, the system generates each codeword vector in the codebook corresponding to the m-th quantization layer, and then determines the m-th level class target tag associated with each codeword vector.

[0087] Please see Figure 6 In some embodiments, the process of generating codeword vectors in the codebook corresponding to the m-th quantization layer and determining the m-th level class target tag associated with each codeword vector includes: the training system clusters the encoding vectors of each sample to obtain K clusters, and determines the cluster center vector corresponding to each of the K clusters. Then, the cluster center vectors corresponding to each of the K clusters are used as codeword vectors in the codebook corresponding to the m-th quantization layer. Furthermore, for each codeword vector, the training system determines the m-th level class target tag associated with that codeword vector based on the m-th level class target tag to which each sample in the cluster to which the codeword vector belongs.

[0088] In some embodiments, the process of clustering the encoding vectors of all samples in the m-th quantization layer of the sample dataset to obtain K clusters can be implemented using the K-means clustering algorithm. The K-means algorithm can cluster the encoding vectors of each sample into compact clusters.

[0089] In some embodiments, for any codeword vector, the training system can statistically analyze the distribution of class tags among all samples in its corresponding cluster and determine the class tag with the highest frequency (or a proportion exceeding a preset threshold) as the class tag associated with that codeword vector. For example, if 90% of the samples in a cluster belong to the "medical" tag, then the codeword vector corresponding to that cluster is labeled as associated with the "medical" tag.

[0090] Please continue reading Figure 6 Taking the second quantization layer as an example, through cluster analysis and label association of the encoded vectors, the training system determines the second-layer class target labels associated with each codeword vector in the codebook B corresponding to the second quantization layer. For example... Figure 6 As shown, code B1 is associated with the "drugs" label, code B2 with the "medical diagnosis" label, code B3 with the "public transport" label, code B4 with the "medical devices" label, and code B5 with the "airplane" label.

[0091] The codebook obtained after clustering (such as codebook B) can simultaneously contain the semantics of sub-labels belonging to multiple different major categories (such as drugs and medical devices under the "medical" category, and buses and airplanes under the "transportation" category). This allows the quantization layer to effectively distinguish and encode sample objects from different fields and categories within a unified feature space.

[0092] In some embodiments, a class-specific tag may correspond to a unique codeword vector in the codebook. For example, if there are only a few samples at level m and the class features are highly concentrated, the "medical" tag may be associated with only one codeword vector.

[0093] Suppose that there are 10 different category tags at level m (such as "medical", "transportation", etc.), and the codebook size is set to 256 (i.e., containing 256 codeword vectors). Forcing a one-to-one correspondence between codewords and tags would waste codeword resources. Furthermore, since the number of samples in each cluster under each codeword vector is enormous, a single codeword vector cannot adequately represent the rich diversity of samples under a given tag.

[0094] Therefore, in some embodiments, one class tag can correspond to multiple codeword vectors in the codebook. The training system can control the number of clusters K in the clustering process (e.g., setting K=200) so that the number of generated codeword vectors is greater than the number of class tags.

[0095] In this scenario, the same target tag (e.g., "medical") might be associated with multiple different codeword vectors (e.g., codeword vectors A1, A20, and A31 are all associated with the "medical" tag). The training system can set a global cluster size K, allowing all samples to freely cluster in the feature space, forming multiple fine-grained clusters associated with the same tag. Alternatively, the training system can set the number of sub-clusters for each target tag individually (e.g., specifying that the "medical" tag must have 20 clusters), thereby forcing that tag to occupy more codeword resources.

[0096] Through the aforementioned one-to-many mapping mechanism (i.e., one class label is associated with multiple codeword vectors), the trained semantic quantization model can utilize multiple fine-grained codeword vectors to jointly represent a class label. This not only improves the utilization rate of the codebook but also allows each codeword vector to focus on capturing subtle features of a specific subset under that label. For example, under the "medical" label, codeword vector A1 might focus on representing features of "liquid medicine," while codeword vector A20 might focus on representing features of "solid medicine." This fine-grained feature capture capability significantly enhances the semantic quantization model's ability to distinguish sample objects and its representational sophistication. Furthermore, through this clustering process, the model can also uncover potential classification structures hidden within the data but not explicitly defined in the original labeling system. For example, under the "transportation" label, clustering might automatically distinguish between "public transportation" and "non-public transportation," corresponding to different codeword vectors. This means that the model can not only learn existing knowledge (labels) but also uncover classification logic outside the labeling system through clustering, thereby further enriching the semantic information contained in the generated identifier sequence.

[0097] S423: The training system determines the target codeword vector to be aligned for each sample based on the codeword vectors in the codebook corresponding to the m-th quantization layer and their associated m-th level class target tags, and determines the semantic alignment loss of the m-th quantization layer based on the target codeword vectors and encoding vectors corresponding to each sample.

[0098] In some embodiments, the target codeword vector refers to the reference vector selected from the codebook of the current quantization layer as the most suitable for representing the semantic features of the current sample. It serves as a supervisory signal for determining the semantic alignment loss function.

[0099] In some embodiments, the process of determining the target codeword vector to be aligned with the encoding vector of each sample includes: for each sample, the training system, based on the m-th level class target tag to which the sample belongs, determines at least one candidate codeword vector associated with the corresponding class target tag from the codebook corresponding to the m-th quantization layer. Then, among the at least one candidate codeword vector, the candidate codeword vector with the highest similarity to the encoding vector of the sample is taken as the target codeword vector.

[0100] In some embodiments, if the target tag of the m-th level class to which the sample belongs corresponds to a unique codeword vector in the codebook, then the at least one candidate codeword vector contains only this unique codeword vector. Therefore, the training system directly determines this unique codeword vector as the target codeword vector, and subsequently calculates the semantic alignment loss based on the target codeword vector and the sample encoding vector.

[0101] In some embodiments, a target tag can correspond to multiple codeword vectors in a codebook. In this case, the at least one candidate codeword vector contains multiple different codeword vectors (e.g., the "medical" tag corresponds to three codewords: A1, A20, and A31). The training system needs to calculate the similarity between the sample's encoding vector and these multiple candidate codeword vectors, and select the most similar one (e.g., A20) as the target codeword vector, thereby achieving more accurate semantic alignment.

[0102] S424: The training system updates the parameters of the encoding network in the m-th quantization layer based at least on the semantic alignment loss.

[0103] Since different quantization layers correspond to different levels of class tags (e.g., the first quantization layer corresponds to the major class, and the second quantization layer corresponds to the subclass), their semantic alignment goals and constraints are logically sequential yet independent. Therefore, the training system can begin training and updating the parameters of the encoding network from the first quantization layer. After obtaining the trained first quantization layer (and fixing its encoding network parameters), the residual vector output by the first quantization layer is used as the input to the second quantization layer for training, and so on. This approach avoids the interference of subsequent layer fluctuations on preceding layers, helping each quantization layer focus on capturing semantic information at a specific level, thereby improving the representation accuracy of identifiers at each level and the convergence stability of the model.

[0104] In some embodiments, for a specific training epoch of the m-th quantization layer, the codeword vectors and their associated class tags in the codebook are fixed. That is, within that epoch, the training system uses this fixed set of codeword vectors as a benchmark to calculate the semantic alignment loss and update the parameters of the encoding network.

[0105] As training progresses to the next round of the m-th quantization layer, the distribution of the encoded vectors generated for the same input vector will change (e.g., become more compact) because the parameters of the encoding network have been optimized and updated in the previous round. Therefore, the training system will re-execute clustering and label association operations based on the newly generated encoded vectors.

[0106] In other words, codebook generation is a dynamic evolutionary process: each round of codebook generation is based on the latest state of the encoding network, and the updated encoding network, in turn, influences the generation of the next round of codebook. Through this iterative cycle, the clustering results become increasingly closer to the class-target tag system, that is, the codeword vectors become increasingly closer to the class-target tag system, thereby achieving the co-evolution of the encoding network and the codebook.

[0107] In some embodiments, the number of samples in the sample dataset is N, and for the i-th sample, where i ranges from 1 to N, the encoding vector of the i-th sample is denoted as . The target codeword vector corresponding to the i-th sample is denoted as . The semantic alignment loss is at least based on and The similarity between them is determined and used to constrain the feature space. Towards near.

[0108] In some embodiments, and The similarity between them can be cosine similarity, which is used by the training system to measure. and Differences in the direction of the feature space.

[0109] By using the semantic alignment loss of the m-th quantization layer, the training system can train the encoding vector of the m-th quantization layer of the semantic quantization model for the i-th sample. In the feature space, the target codeword vector is associated with the target tag of the class to which the i-th sample belongs as much as possible. Proximity. For example, when the class tag of a sample at the m-th level belongs to "traffic", the semantic alignment loss will constrain the encoding vector output by the m-th quantization layer for the i-th sample. Make it closer to the target codeword vector associated with "traffic". .

[0110] As mentioned earlier, if the target signature of this sample at the m-th level is associated with only one codeword vector, then that codeword vector is the target codeword vector. Semantic alignment loss constraint To the unique target codeword vector near.

[0111] For cases where a sample is associated with multiple codeword vectors at the m-th level of the target category (e.g., the "medical" tag corresponds to three codewords A1, A20, and A31): if the i-th sample is determined to be most similar to A20 through the aforementioned matching process, then the codeword vector of A20 is the target codeword vector. At this point, semantic alignment loss is used to constrain... Move closer to the target codeword vector of A20's codeword vector.

[0112] Please continue reading Figure 6Taking the second quantization layer as an example, the process of determining the target codeword vector and updating parameters is explained. Let the second-level class target label of the i-th sample object be "medical device". The training system traverses each codeword vector and its associated label in codebook B, as shown in the figure. Codeword B1 is associated with the label "drug", codeword B2 with the label "medical diagnosis", codeword B3 with the label "public transport", codeword B4 with the label "medical device", and codeword B5 with the label "airplane". The training system determines that the label associated with codeword B4 is consistent with the sample label, therefore, the codeword vector of codeword B4 is determined as the target codeword vector for this sample (if there are multiple codewords associated with "medical device", the one with the highest similarity is selected). Then, the training system calculates the difference between the encoded vector of this sample (output by the second-layer encoding network) and the target codeword vector B4 to determine the semantic alignment loss. Based on this semantic alignment loss, the training system updates the parameters of the second-layer encoding network.

[0113] In some embodiments, the codebook, excluding the target codeword vector The codeword vectors other than are denoted as The semantic alignment loss is also based on and The similarity between them is determined and used to constrain the feature space. keep away .

[0114] Here, other codeword vectors are introduced. (j takes values ​​from 1 to K, and j is not equal to i) is to construct a dynamic exclusion mechanism based on semantic relevance, aiming to allow Avoid codeword vectors that are associated with tags whose semantics differ significantly from the target tag of the current sample class, while maintaining a certain tolerance for codeword vectors that are associated with tags with similar semantics.

[0115] Assume the target codeword vector of the i-th sample Associate with the "medical device" tag. If a certain non-target codeword vector... The associated tag is "medical consumables". Because "medical consumables" and "medical devices" are semantically similar, the training system will reduce the bias towards "medical devices" by adjusting the weighting coefficient. and A penalty is applied based on the similarity between the components. This means that even if the model makes a small deviation during semantic quantization, the penalty that should have been assigned to the model will be reduced. The samples were incorrectly assigned to Nearby, because the two are semantically similar, this error is acceptable in the output, so the model will not force them to be too far apart. Conversely, if a certain non-target codeword vector... The associated tag is "transportation," and due to its significant semantic differences, the training system imposes a strong penalty, forcing... As far away as possible from the feature space This ensures that serious classification errors across major categories do not occur.

[0116] To accurately quantify the above-mentioned constraint objectives, in some embodiments, the semantic alignment loss of the m-th quantization layer is... It is determined based on the following formula (1): (1) in, The weight coefficients are determined based on the semantic relevance between the i-th sample and the j-th sample on the class target tag at the m-th level. This represents the number of codeword vectors in the codebook corresponding to the m-th quantization layer.

[0117] Introduced in the formula The summed loss value is then normalized, therefore This represents the average semantic alignment loss calculated by the m-th quantization layer for all samples in the current batch. Averaging eliminates the influence of sample size on the loss value, ensuring the stability of parameter updates under different sample sizes and reducing the impact of sample size fluctuations on model training performance.

[0118] The molecular part of the semantic alignment loss mentioned above , used to measure the encoding vector of the i-th sample Its target codeword vector The similarity between them. In minimizing the loss, the model maximizes the value of this numerator, thereby constraining... Towards Proximity is achieved to semantically align the encoded vector to the target codeword vector.

[0119] The denominator of the semantic alignment loss mentioned above also introduces... , used to measure the encoding vector of the i-th sample Other non-target codeword vectors Similarity in the feature space, and to achieve differentiated similarity constraints, dynamic weight coefficients are further introduced. This enables dynamic adjustment of the item.

[0120] In some embodiments, Determined based on the following formula (2): (2) in, Codeword vector The semantic vector of the associated m-th level class target tag; Codeword vector The semantic vector of the associated m-th level class target tag; express and Semantic relevance between them This is a hyperparameter.

[0121] In some embodiments, By means of The text of the associated m-th level class target tag is vectorized to obtain the vector. If the associated m-th level class target is labeled "medical", then The semantic encoding model is pre-trained to input the term "medical" into it. This model extracts features from the textual descriptions of the target tags to obtain their semantic vectors. The training system can obtain corresponding semantic vectors with general semantic information by inputting target tags (e.g., "transportation," "medical") into the semantic encoding model.

[0122] Here, Essentially, it acts as a penalty coefficient, and its value is negatively correlated with the semantic relevance between tags. Through this... Implement the following two control strategies. On the one hand, it can be done through Achieved We maintain a certain tolerance for codeword vectors associated with similar semantic tags. Assume the target codeword vector... (For example, B1) is associated with the "medicine" tag, while a certain non-target codeword vector For example, (B20) is also associated with the "medicine" label (or the semantically very similar "medical diagnosis"). In this case, and semantic relevance between Close to 1 (or very high).

[0123] At this point, the weighting coefficients are calculated according to formula (2). It will approach 0 (or be very small). Returning to the denominator of formula (1), the smallest... It will significantly inhibit This is the contribution of this term to the total loss. This means that even the encoded vector... and They are very close in the feature space (i.e.) Even if the size is large, the model will not be severely penalized. This mechanism allows for the encoding vector... In the feature space, it is placed near other codeword vectors that are consistent with or similar to the target semantics.

[0124] On the other hand, it can be done through Achieved Avoid codeword vectors associated with tags whose semantics differ significantly from the target tag of the current sample class. Assume the target codeword vector... (For example, A1) is associated with the "medical" tag, while a certain non-target codeword vector (For example, A15) Associated with the "Transportation" label. At this point, and semantic relevance between Lower.

[0125] The weighting coefficients are calculated according to formula (2). It will be a large positive value. Returning to the denominator of formula (1), the larger... It will be magnified This contributes to the total loss. To reduce the total loss, the model must force... The value decreases. Because... It is a fixed codeword vector; the model can only change the output codeword vector by adjusting the encoding network parameters. , so that it is with The similarity decreases (i.e., the distance increases).

[0126] Through this strong rejection mechanism, the model can effectively prevent the encoded vector from being blocked. The erroneous drift to the vicinity of codeword vectors with huge semantic differences results in a large distance between different semantic categories in the feature space.

[0127] The discrete nature of quantization layer operations in traditional RQ-VAE leads to a discontinuous loss function. When a sample object changes its matched codeword due to a small perturbation during training, it causes drastic jumps in the loss value, resulting in loss oscillations and hindering convergence. This specification's embodiments introduce a dynamic weight coefficient-based approach... The soft alignment operation constructs a smooth semantic alignment loss. Under the constraint of this semantic loss function, even if the matching result of the sample shifts to a nearby codeword... ,because By assigning lower exclusion weights to semantically similar codewords, the loss values ​​change gradually rather than abruptly. This smooth change allows the training system to update parameters more stably and efficiently, improving the model's convergence speed and stability on large-scale datasets.

[0128] To reduce the loss of semantic information in the descriptive information of the generated identifier sequences, the training system can be evaluated and constrained by reconstruction loss.

[0129] In some embodiments, each iteration of the m-th quantization layer further includes: for each sample, the training system performs semantic quantization on the encoding vectors of the sample from the 1st to the m-th quantization layers to output m codeword vectors, and decodes the m codeword vectors to obtain a reconstructed vector. The training system determines a reconstruction loss based on the difference between the embedding vector corresponding to the description information of the sample object and the reconstructed vector. Updating the parameters of the encoding network in the m-th quantization layer includes: the training system constructs a target loss based on the reconstruction loss and the semantic alignment loss, and then updates the parameters of the encoding network in the m-th quantization layer based on the target loss.

[0130] In some embodiments, the training system can aggregate the m codeword vectors output from the first to the mth quantization layers to obtain an aggregated vector corresponding to the m codeword vectors. Then, the training system can decode the aggregated vector using a decoding network to obtain the reconstructed vector.

[0131] Please see Figure 7 Taking the iteration of the second quantization layer as an example, for each sample, the training system obtains the codeword vector output by the first quantization layer for that sample (e.g., the codeword vector corresponding to A2) and the codeword vector output by the second quantization layer for that sample (e.g., the codeword vector corresponding to B4). Subsequently, the training system aggregates these two codeword vectors and inputs the aggregated vector into the decoding network to obtain the reconstruction vector for the current layer.

[0132] In some embodiments, the training system can obtain the reconstruction loss by calculating the mean squared error (MSE) between the embedding vector corresponding to the description information of the sample object and the reconstructed vector. The reconstruction loss is used to constrain the semantic quantization model to improve the similarity between the information contained in the generated aggregate vector and the description information of the input sample object.

[0133] In some embodiments, target loss It can be based on reconstruction loss Semantic alignment loss with the first quantization layer The weighted sum is obtained. The training system can set corresponding weight coefficients according to the actual application scenario and the degree of emphasis on the two losses.

[0134] In some embodiments, based on the constructed target loss, the training system updates the parameters of the encoding network in the m-th quantization layer. This enables the trained semantic quantization model to achieve two objectives: it can output an aggregated vector with hierarchical semantic information through semantic alignment loss, and it can enhance the similarity between the information carried by the aggregated vector and the descriptive information through reconstruction loss.

[0135] In some embodiments, a layer normalization loss to balance the uniform use of codeword vectors may be added to the target loss. Each iteration of the m-th quantization layer further includes: the training system determining the layer normalization loss of the m-th quantization layer, wherein the layer normalization loss characterizes the degree to which each codeword vector in the codebook is used uniformly. Constructing the target loss includes: the training system constructing the target loss based on the reconstruction loss, the semantic alignment loss, and the layer normalization loss.

[0136] However, during training, certain target categories (e.g., "medical devices") may have a large number of object samples. Therefore, codeword vectors associated with these target categories may attract too many samples, leading to overuse of the corresponding codeword vectors, while other codeword vectors may remain idle for extended periods (i.e., "dead codes"). This uneven distribution reduces the effective capacity of the codebook and limits the model's ability to represent long-tail categories. Therefore, by introducing layer normalization loss as an auxiliary regularization term, the model can be trained to use codeword vectors in the codebook as evenly as possible while maintaining semantic alignment.

[0137] In some embodiments, the layer normalization loss It is determined based on the following formula (3): (3) in, The frequency of use of the k-th codeword vector in the codebook. This is a hyperparameter.

[0138] The above formula represents the variance of the frequency distribution of each codeword vector in the codebook. When all codeword vectors are used at exactly the same frequency (i.e....), When the variance is 0, the layer normalization loss is minimized, indicating that the codeword vectors are used evenly and in an equilibrium state. However, when the frequency distribution of codeword vector usage is uneven (e.g., some codeword vectors are used very frequently while others are idle), the variance is large, leading to… Larger.

[0139] Considering that the sample distribution may differ significantly between different training batches during training based on mini-batch samples, leading to large fluctuations in codeword allocation statistics, it is difficult to accurately capture the long-term trend of codeword usage by directly using the raw count statistics of the current batch. In some embodiments, the training system introduces an Exponentially Weighted Moving Average (EMA) mechanism to maintain the smooth usage frequency of each codeword vector in the m-th quantization layer online.

[0140] For the k-th codeword vector, the training system performs EMA updates based on its usage frequency in the current batch, combined with historical statistical values, to obtain its smoothed usage frequency. ( The frequency of use of the k-th identifier for the m-th quantized layer is used as a reference. Compared to the raw count that only depends on the current batch, the smoothed frequency of use obtained through the EMA mechanism has better stability and cross-batch comparability, and can more accurately reflect the true load of the identifier during long-term training. This allows the layer normalization loss calculated based on this frequency to more robustly guide model parameter updates.

[0141] The above process is based on joint constraints of three objectives: reconstruction loss, semantic alignment loss, and layer normalization loss. However, directly performing multi-objective joint optimization may increase the difficulty of model convergence. Furthermore, during layer-by-layer quantization and residual propagation, as the number of quantization layers increases, the input vectors of subsequent quantization layers are residual vectors after multiple semantic stripping processes, and the semantic information they contain gradually decreases. The semantic information contained in the residual vectors may only include texture noise in the image or non-keyword information in the text. At this point, the semantic quantization model struggles to effectively semantically align these residual vectors with specific text class targets. If the training system forces these low-semantic-information residual vectors to align with hierarchical class targets, it may lead to the model establishing incorrect semantic mappings, thereby damaging the model's training performance.

[0142] Therefore, the training system can focus the constraints of the semantic alignment loss on the preceding quantization layers to avoid unreasonable semantic alignment in other quantization layers from damaging the overall training effect of the model. However, for subsequent quantization layers, if there is a lack of effective constraints, the "codebook collapse" phenomenon can easily occur, that is, a large number of residual vectors are mapped to only a few identifiers, resulting in extremely low utilization of identifiers and failure to effectively encode subtle features in the residuals.

[0143] Please see Figure 8 After the semantic stripping process of the first three quantization layers, the semantic information contained in the third residual vector input to the fourth quantization layer is already very sparse. At this point, if the identifier output by the fourth quantization layer is forced to align with the semantic representation of a more specific fine-grained label (such as "model A"), the residual noise can easily cause the model to establish an incorrect semantic mapping, thus destroying the feature representation.

[0144] If no effective constraints are imposed on the fourth quantization layer, relying solely on reconstruction loss constraints, the distribution of usage frequency of candidate identifiers associated with this layer often exhibits an unbalanced state. For example... Figure 8As shown in the statistical chart, the fourth quantization layer may tend to map the vast majority of residual vectors to a few identifiers (such as D1 and D2), while other identifiers (such as D3-D5) are used very infrequently, resulting in low codebook utilization and an inability to effectively encode the diverse details in the residuals.

[0145] Therefore, this specification also provides a strategy for training different loss functions for different quantization layers, that is, focusing on different optimization objectives: the first m-th quantization layer focuses on training semantic alignment capability; the last Mm quantization layers focus on training the uniformity of codeword vector usage to improve the detail richness of residual coding.

[0146] In some embodiments, the training system has an nth quantization layer among the M quantization layers, where n is an integer less than or equal to M, and each iteration of the nth quantization layer includes steps S510-S540.

[0147] S510: For each sample in the sample dataset, the training system sequentially passes the description information of the sample object from the first quantization layer to the nth quantization layer, and obtains the input vector of the sample in the nth quantization layer.

[0148] In some embodiments, the nth quantization layer is located after the mth quantization layer (i.e., n>m). That is, the nth quantization layer belongs to the subsequent layer in the model quantization layer and is mainly responsible for processing the residual information remaining after semantic stripping of the preceding layers (including the mth quantization layer).

[0149] When performing the training iteration of the nth quantization layer, the previous layers have completed parameter updates and are in a stable state. In order to obtain the input vector of the nth quantization layer, the training system first processes the samples using the first n-1 quantization layers that have been trained and have fixed parameters.

[0150] Please see Figure 8 This means that the residual vector entering the fourth quantization layer (i.e., n=4) is calculated based on the output of the preceding layers, which has already been fixed. For example, after the semantic alignment training described above, the first three quantization layers can stably output a sequence of identifiers (such as A2, B4, C2) that highly correspond to the semantic representation of the class tags. Therefore, the input vector received by the nth quantization layer is essentially the pure residual remaining after stripping away the explicit semantic information of the first n-1 layers, providing a stable input foundation for the nth layer to focus on detail encoding.

[0151] S520: The training system uses the encoding network of the nth quantization layer to encode the input vector of each sample, and obtains the encoding vector of each sample. Based on the clustering of the encoding vectors of each sample, it generates the codeword vectors in the codebook corresponding to the nth quantization layer.

[0152] It should be noted that for the nth quantization layer (usually a post-quantization layer), since the semantic information in its input vector is already relatively sparse, the training system can use unsupervised clustering algorithms to mine the inherent distribution structure of the residual data when generating the codebook. At this point, it is no longer necessary to force the codeword vectors to be associated with specific class tags. The training system clusters the encoding vectors of all samples at the nth layer and uses the resulting cluster center vectors as codeword vectors.

[0153] S530: The training system quantizes the encoding vector of each sample based on the codeword vector in the codebook corresponding to the nth quantization layer, outputs the codeword vector, and determines the layer normalization loss of the nth quantization layer based on the semantic quantization of each sample, wherein the layer normalization loss characterizes the degree to which each codeword vector in the codebook is used evenly.

[0154] Layer normalization loss is used to constrain the nth quantization layer to use all associated codeword vectors as evenly as possible when processing sample objects, avoiding the concentrated use of a few. The training system calculates the frequency of each codeword vector in the codebook being selected in the current training round by counting the frequency. Then, the variance of this frequency distribution is calculated using the aforementioned formula (3). At this stage, the training system no longer calculates the semantic alignment loss (or sets its weight to 0), because the main goal of the nth layer is to solve the "codebook collapse" problem. By focusing on minimizing the layer normalization loss, the training system makes the nth quantization layer evenly map the incoming residual vectors onto each codeword in the codebook.

[0155] S540: The training system updates the parameters of the encoding network in the nth quantization layer based at least on the layer normalization loss.

[0156] This targeted update strategy can improve the coding efficiency and discriminability of the subsequent quantization layer without destroying the semantic structure of the preceding layer.

[0157] Please refer to the following formula (4), which uses the Sigmoid function to achieve a smooth transition between different quantization layers for semantic loss and layer normalization loss.

[0158] (4) in, Indicates the current quantization layer number. and These are hyperparameters used to control the starting position and transition smoothness of the loss activation. This is the Sigmoid function. The coefficient is in... When the time is close to 0, it has almost no regularizing effect on shallow layers; when The time gradually rises and approaches 1, making It plays a full role in the deep layers.

[0159] According to this formula, for the preceding quantization layers, the Sigmoid function value approaches 0, making the model primarily constrained by semantic alignment loss. As the layers deepen, the Sigmoid function value gradually increases and approaches 1, thereby automatically strengthening the constraint of the layer standardization loss. This mechanism avoids the truncation effect caused by manually and rigidly dividing the quantization layers, enabling the model to smoothly switch feature learning objectives between layers.

[0160] Furthermore, while constraining the nth quantization layer based on layer normalization loss, the training system can also simultaneously utilize reconstruction loss to jointly optimize the quantization layer.

[0161] Please continue reading Figure 8 After training constrained by layer normalization loss, the frequency of codeword vector usage in the fourth quantization layer changes significantly. The residual vectors are mapped more evenly across multiple codeword vectors, from D1 to D5, avoiding the concentration only on D1 and D2. This even distribution significantly improves the effective capacity of the fourth quantization layer's codebook, enabling it to capture and retain more subtle feature differences.

[0162] For example, before the model parameters are updated, regardless of the specific IV drip model, the final identifier sequence generated by the model may be highly similar, such as always outputting (A2, B4, C2, D1) or (A2, B4, C2, D2). This means that the model loses the key information to distinguish different models, making it difficult for subsequent retrieval or recommendation systems to differentiate specific products.

[0163] However, after being constrained by the loss function, the model can fully utilize the idle resources in the codebook, mapping different models (or different batches, subtle appearance differences) of IV drips to different codeword vectors, such as generating sequences with obvious differences like (A2, B4, C2, D3) and (A2, B4, C2, D5). This allows the generated identifier sequences to maintain semantic consistency in the preceding parts (both are A2, B4, C2), while also possessing fine-grained discriminative ability at the end of the identifier sequence.

[0164] It should be noted that, for the sake of simplicity, only 5 codeword vectors are shown here. In practical applications, the codebook associated with each quantization layer usually contains a much larger number of codeword vectors (e.g., 256). Under the constraint of layer normalization loss, this large-scale uniform mapping will bring more significant discrimination and information encoding capabilities.

[0165] In summary, the semantic quantization model training method provided in this specification adopts a layer-by-layer iterative training approach for M sequentially connected quantization layers. It utilizes multi-level class target tags to guide the generation of codebooks and semantic quantization for each quantization layer, with each quantization layer configured with an independent encoding network. Each iteration of the m-th quantization layer includes: the training system encodes the input vector of each sample using the encoding network of that quantization layer to obtain the encoding vector of each sample; it then generates a codebook based on the clustering of the m-th level class target tag to which each sample belongs and the encoding vector; and determines that the generated codeword vector is associated with the semantics of the tag category. Furthermore, the training system determines the target codeword vector to be aligned for each sample's encoding vector, and based on the target codeword vector and encoding vector corresponding to each sample, determines the semantic alignment loss of the m-th quantization layer to update the parameters of the encoding network of that layer. This specification, through layer-by-layer iteration and the design of independent encoders, decouples the training of each quantization layer, solving the problem of severe semantic decay caused by a single encoding network's inability to adapt to multi-level features, allowing the quantization network to focus on capturing the semantic information of that level. By generating and determining target codeword vectors based on class-target signatures, explicit supervision information is provided to the model. This supervision not only enables the encoded vectors to converge to the target codeword vectors to be aligned more quickly, improving the training convergence speed, but also avoids the semantic drift problem that easily occurs in unsupervised clustering. Furthermore, based on the aforementioned semantic alignment loss, the identifier sequence generated by the trained model is no longer composed of meaningless random symbols, but rather consists of identifiers carrying semantic information, enabling the identifier sequence to reflect the differences in category affiliation between different objects. The semantic quantization model trained in this way can generate identifier sequences with semantic representation and discriminative power. This improves the ability of systems applying this identifier (such as retrieval or recommendation systems) to distinguish different objects and the accuracy of retrieval recall based on this identifier.

[0166] This specification, in another aspect, provides a computer-readable non-transitory storage medium storing at least one instruction set for training a semantic quantization model. When the at least one instruction set is executed by a processor, it instructs the processor to implement the steps of the semantic quantization model training method P400 described herein, or the steps of the semantic identifier generation method P300 described herein. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing system 200, the program code causes the computing system 200 to execute the steps of the semantic quantization model training method P400 described herein. The program product for implementing the above methods may employ a portable compact disk read-only memory (CD-ROM) containing program code and may run on the computing system 200. However, the program product of this specification is not limited thereto. In this specification, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a readable storage medium include: portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include a data signal propagated as part of a carrier wave in baseband, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages.The program code can be executed entirely on computing system 200, partially on computing system 200, as a standalone software package, partially on computing system 200 and partially on a remote computing system, or entirely on a remote computing system.

[0167] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0168] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0169] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0170] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and to aid in understanding a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art, upon reading this specification, may readily identify some of the devices as separate embodiments. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. And the content of each secondary embodiment is valid even if it contains fewer than all the features of a single foregoing disclosed embodiment.

[0171] Every patent, patent application, publication of a patent application, and other material such as articles, books, specifications, publications, documents, articles, etc., cited herein, except for those inconsistent with or conflicting with this document, or those having a restrictive effect on the widest scope of the claims, may be incorporated herein by reference for all purposes now or hereafter associated with this document. Furthermore, in the event of any inconsistency or conflict between the description, definition, and / or use of relevant terms in any material and the description, definition, and / or use of relevant terms in this document, the terms in this document shall prevail.

[0172] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A method for training a semantic quantization model, the semantic quantization model comprising M sequentially connected quantization layers, where M is an integer greater than 1, the method comprising: Obtain a sample dataset, wherein each sample in the sample dataset includes: descriptive information of the sample object and its corresponding multi-level class target tags; and Following the connection order of the M quantization layers, each quantization layer is iteratively trained using the sample dataset, wherein at least one m-th quantization layer exists among the M quantization layers, where m is an integer less than M, and each iteration of the m-th quantization layer includes: For each sample in the sample dataset, the description information of the sample object is passed sequentially from the first quantization layer to the mth quantization layer, and the input vector of the sample in the mth quantization layer is obtained. The input vector of each sample is encoded using the encoding network of the m-th quantization layer to obtain the encoded vector of each sample. Based on the m-th level class target tag to which each sample belongs and the clustering of the encoded vectors of each sample, codeword vectors in the codebook corresponding to the m-th quantization layer are generated, and the m-th level class target tag associated with each codeword vector is determined. Based on the codeword vectors in the codebook corresponding to the m-th quantization layer and their associated m-th level class target tags, the target codeword vectors to be aligned for each sample's encoding vector are determined. Based on the target codeword vectors and encoding vectors corresponding to each sample, the semantic alignment loss of the m-th quantization layer is determined. The parameters of the encoding network in the m-th quantization layer are updated based at least on the semantic alignment loss.

2. The method according to claim 1, wherein, The sample dataset contains N samples. For the i-th sample, where i ranges from 1 to N, the encoding vector of the i-th sample is denoted as... The target codeword vector corresponding to the i-th sample is denoted as . ; The semantic alignment loss is at least based on and The similarity between them is determined and used to constrain the feature space. Towards near.

3. The method according to claim 2, wherein, The codebook contains, except for the target codeword vector The codeword vectors other than are denoted as The semantic alignment loss is also based on and The similarity between them is determined and used to constrain the feature space. keep away .

4. The method according to claim 3, wherein, The semantic alignment loss It is determined based on the following formula: in, Based on codeword vectors The associated m-th level class target signature and codeword vector The weight coefficients determined by the associated m-th level class target tag. This represents the number of codeword vectors in the codebook corresponding to the m-th quantization layer.

5. The method according to claim 4, wherein, Determined based on the following formula: in, Codeword vector The semantic vector of the associated m-th level class target tag; Codeword vector The semantic vector of the associated m-th level class target tag; express and Semantic relevance between them This is a hyperparameter.

6. The method according to claim 1, wherein, Each iteration of the m-th quantization layer also includes: For each sample, semantic quantization is performed on the encoding vector of the sample by the first quantization layer to the mth quantization layer to output m codeword vectors, and the m codeword vectors are decoded to obtain the reconstructed vector; The reconstruction loss is determined based on the difference between the embedding vector corresponding to the description information of the sample object and the reconstruction vector; The step of updating the parameters of the encoding network in the m-th quantization layer based at least on the semantic alignment loss includes: Based on the reconstruction loss and the semantic alignment loss, a target loss is constructed; Based on the target loss, the parameters of the encoding network in the m-th quantization layer are updated.

7. The method according to claim 6, wherein, Each iteration of the m-th quantization layer further includes: determining the layer normalization loss of the m-th quantization layer, wherein the layer normalization loss characterizes the degree to which each codeword vector in the codebook is used evenly; The construction of the target loss based on the reconstruction loss and the semantic alignment loss includes: The target loss is constructed based on the reconstruction loss, the semantic alignment loss, and the layer normalization loss.

8. The method according to claim 7, wherein, The layer normalization loss It is determined based on the following formula: in, The frequency of use of the k-th codeword vector in the codebook. This is a hyperparameter.

9. The method according to claim 1, wherein, Among the M quantization layers, there is also an nth quantization layer, where n is an integer less than or equal to M. Each iteration of the nth quantization layer includes: For each sample in the sample dataset, the description information of the sample object is passed sequentially from the first quantization layer to the nth quantization layer, and the input vector of the sample in the nth quantization layer is obtained. The input vector of each sample is encoded using the encoding network of the nth quantization layer to obtain the encoded vector of each sample. Based on the clustering of the encoded vectors of each sample, the codeword vectors in the codebook corresponding to the nth quantization layer are generated. Based on the codeword vectors in the codebook corresponding to the nth quantization layer, the encoding vector of each sample is quantized to output a codeword vector. The layer normalization loss of the nth quantization layer is determined based on the semantic quantization of each sample. The layer normalization loss characterizes the degree to which each codeword vector in the codebook is used evenly. The parameters of the encoding network in the nth quantization layer are updated based at least on the layer normalization loss.

10. The method according to claim 10, wherein, The layer normalization loss It is determined based on the following formula: in, The frequency of use of the k-th codeword vector in the codebook. This is a hyperparameter.

11. The method according to claim 10, wherein, The nth quantization layer is located after the mth quantization layer.

12. The method according to claim 1, wherein, When m=1, the input vector is the embedding vector corresponding to the description information of the sample object; When m>1, the input vector is the residual vector between the encoding vector of the (m-1)th quantization layer and the codeword vector output by the (m-1)th layer.

13. The method according to claim 1, wherein, Based on the m-th level class target tag to which each sample belongs and the clustering of the encoding vectors of each sample, codeword vectors in the codebook corresponding to the m-th quantization layer are generated, and the m-th level class target tag associated with each codeword vector is determined, including: The encoded vectors of each sample are clustered to obtain K clusters, and the cluster center vectors corresponding to the K clusters are determined. The cluster center vectors corresponding to the K clusters are respectively used as codeword vectors in the codebook corresponding to the m-th quantization layer; and For each codeword vector, based on the m-th level class target tag to which each sample in the cluster corresponding to the codeword vector belongs, the m-th level class target tag associated with the codeword vector is determined.

14. The method according to claim 13, wherein, Based on each codeword vector in the codebook corresponding to the m-th quantization layer and its associated m-th level class target tag, the target codeword vector to be aligned for each sample's encoding vector is determined, including: For each sample, based on the m-th level class target tag to which the sample belongs, at least one candidate codeword vector associated with the corresponding class target tag is determined from the codebook corresponding to the m-th quantization layer; The candidate codeword vector with the highest similarity to the encoding vector of the sample among the at least one candidate codeword vectors is taken as the target codeword vector.

15. The method according to claim 1, wherein, The sample objects are services or goods; The descriptive information of the sample object includes descriptive information of at least one modality among text, image, or voice.

16. A method for generating a semantic identifier, the method comprising: Obtain the description information of the target object; A trained semantic quantization model is obtained, which includes M sequentially connected quantization layers. Each quantization layer includes an encoding network and a codebook, wherein the codebook includes multiple codeword vectors, and M is an integer greater than 1. The description information of the target object is input into the semantic quantization model, and layer-by-layer quantization and residual propagation are performed through the M quantization layers to obtain codeword identifiers output by each quantization layer. For any m-th quantization layer, the quantization process includes: encoding the input vector using the encoding network of the m-th quantization layer to obtain an encoded vector; semantically quantizing the encoded vector into a codeword vector in the codebook based on the codebook of the m-th quantization layer; and outputting a codeword identifier to identify the corresponding codeword vector. Based on the codeword identifiers output by the M quantization layers respectively, an identifier sequence for identifying the target object is generated.

17. A computing system, comprising: At least one storage medium storing at least one instruction set; as well as At least one processor is communicatively connected to the at least one storage medium, wherein, when the computing system is running, the at least one processor reads the at least one instruction set and executes the method as described in any one of claims 1-15, or executes the method as described in claim 16, according to the instructions of the at least one instruction set.

Images