Generated recommendation semantic label generation method and item recommendation method

Abstract

The application relates to a generative recommendation semantic identification generation method and an item recommendation method, wherein the generative recommendation semantic identification generation method comprises the following steps: acquiring an item representation matrix of an item to be recommended; inputting the item representation matrix into a trained semantic identification generation model to obtain an item semantic identification generated by the semantic identification generation model; wherein the semantic identification generation model is constructed by a neural network based on a residual quantization variational autoencoder; a loss function of the semantic identification generation model is a loss function optimized by a spectral softening alignment mechanism for each L2 norm loss part in the neural network based on the residual quantization variational autoencoder; and the spectral softening alignment mechanism is used for amplifying the gradient weight of the secondary spectral component of the processed characteristic value, thereby improving the generation quality of the semantic identification.

Description

Technical Field

[0001] This application relates to the field of generative recommendation, and in particular to generative recommendation semantic tag generation methods and item recommendation methods. Background Technology

[0002] Generative Recommendation (GR) is a recommendation paradigm that models the recommendation task as a sequence generation problem. Unlike traditional discriminative recommendation models, generative recommendation does not treat recommendations as classification or ranking problems, but rather generates a sequence of identifiers for the next item a user might interact with, thereby achieving recommendation prediction. As an upstream task in generative recommendation, the quality of the generated semantic identifiers (IDs) of items can influence the recommendation performance of downstream generative recommendation models.

[0003] Currently, semantic ID generation methods are mainly divided into two categories: one is the neural network method based on Residual-Quantized Variational Autoencoder (RQ-VAE); the other is the static clustering method based on Residual-Quantized KMeans (RQ-KMeans). RQ-VAE uses a neural network encoder to map item content representations into continuous latent variables and discretizes them into hierarchical semantic IDs through multi-level residual quantization; while RQ-KMeans directly performs hierarchical clustering of item representations, using the cluster center index as the semantic ID. In practical applications, a counterintuitive experimental phenomenon has been observed: the semantic IDs generated by the RQ-VAE method, which has stronger representation learning capabilities, often perform worse in downstream recommendation tasks than those generated by the simpler static clustering method RQ-KMeans. This limits the discriminative power and expressiveness of the semantic IDs ultimately generated by RQ-VAE, resulting in lower semantic generation quality.

[0004] There is currently no effective solution to the problem of low quality in semantic ID generation in related technologies. Summary of the Invention

[0005] This embodiment provides a generative recommendation semantic identifier generation method and an item recommendation method to solve the problem of low quality of semantic ID generation in related technologies.

[0006] Firstly, this embodiment provides a generative recommendation semantic tag generation method, including:

[0007] Obtain the item representation matrix of the items to be recommended;

[0008] The item representation matrix is ​​input into the trained semantic identifier generation model to obtain the item semantic identifier generated by the semantic identifier generation model; wherein:

[0009] The semantic identifier generation model is constructed from a neural network based on residual quantization variational autoencoder; the loss function of the semantic identifier generation model is the loss function after optimizing each L2 norm loss part in the neural network based on residual quantization variational autoencoder using a spectral softening alignment mechanism; the spectral softening alignment mechanism is used to amplify the gradient weights of the secondary spectral components of the processed feature values.

[0010] In some embodiments, before inputting the item representation matrix into the trained semantic tag generation model to obtain the item semantic tags generated by the semantic tag generation model, the method further includes:

[0011] The loss function of the neural network based on residual quantization variational autoencoder is optimized based on the spectral softening alignment mechanism, and the model is trained to obtain the trained semantic label generation model.

[0012] In some embodiments, the loss function of the neural network based on the residual quantization variational autoencoder is optimized based on the spectral softening alignment mechanism, and the model is trained to obtain the trained semantic label generation model, including:

[0013] Singular value decomposition is performed on the target representation matrix of the L2 norm loss part to be optimized, and spectral components are extracted from the results of the singular value decomposition.

[0014] The softening coefficient for the current training round is determined based on the S-curve function and learnable parameters.

[0015] The intensity of the spectral component is softened based on the softening coefficient to obtain the target spectral component;

[0016] The L2 norm loss component is replaced based on the target spectral components to obtain the loss function optimized by the spectral softening alignment mechanism.

[0017] In some embodiments, the L2 norm loss component is replaced based on the target spectral component to obtain a loss function optimized by the spectral softening alignment mechanism, including:

[0018] When the L2 norm loss to be optimized is the reconstruction loss, the reconstruction loss is replaced with the alignment loss between the target spectral component of the sample item representation matrix and the predicted representation matrix; the predicted representation matrix is ​​the item representation matrix predicted by the neural network based on the residual quantization variational autoencoder in the current training round.

[0019] In some embodiments, the L2 norm loss component is replaced based on the target spectral component to obtain a loss function optimized by the spectral softening alignment mechanism, including:

[0020] When the L2 norm loss to be optimized is the codebook loss, the codebook loss is replaced with the alignment loss between the target spectral component of the input residual representation and the codebook representation.

[0021] In some embodiments, the L2 norm loss component is replaced based on the target spectral component to obtain a loss function optimized by the spectral softening alignment mechanism, including:

[0022] When the L2 norm loss to be optimized is the commitment loss, the commitment loss is replaced with the alignment loss between the target spectral component represented by the codebook and the residual representation.

[0023] Secondly, this embodiment provides an item recommendation method, the method comprising:

[0024] The generative recommendation semantic identifier generation method provided in the first aspect above is executed to determine the item semantic identifier of the item to be recommended;

[0025] Based on the semantic identifiers of the items and the user's historical behavior information, combined with the trained generative recommendation model, the item recommendation results are obtained.

[0026] The third aspect is that this embodiment provides a generative recommendation semantic tagging generation device, including: an acquisition module and a generation module; wherein:

[0027] The acquisition module is used to acquire the item representation matrix of the item to be recommended;

[0028] The generation module is used to input the item representation matrix into the trained semantic identifier generation model to obtain the item semantic identifier generated by the semantic identifier generation model; wherein:

[0029] The semantic identifier generation model is constructed from a neural network based on residual quantization variational autoencoder; the loss function of the semantic identifier generation model is the loss function after optimizing each L2 norm loss part in the neural network based on residual quantization variational autoencoder using a spectral softening alignment mechanism; the spectral softening alignment mechanism is used to amplify the gradient weights of the secondary spectral components of the processed feature values.

[0030] Fourthly, this embodiment provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the generative recommendation semantic tag generation method described in the first aspect, or the item recommendation method described in the second aspect.

[0031] Fifthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the generative recommendation semantic tag generation method described in the first aspect, or the item recommendation method described in the second aspect.

[0032] Compared with related technologies, this embodiment provides a generative recommendation semantic tag generation method and an item recommendation method. The generative recommendation semantic tag generation method obtains the item representation matrix of the item to be recommended; inputs the item representation matrix into a trained semantic tag generation model to obtain the item semantic tag generated by the semantic tag generation model; wherein: the semantic tag generation model is constructed from a neural network based on residual quantization variational autoencoder; the loss function of the semantic tag generation model is a loss function optimized by applying a spectral softening alignment mechanism to each L2 norm loss component in the residual quantization variational autoencoder neural network; the spectral softening alignment mechanism is used to amplify the gradient weights of the secondary spectral components of the processed feature values. It can amplify the gradient weights of the secondary spectral components of the processed feature values, thereby improving the generation quality of the semantic tag.

[0033] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0034] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0035] Figure 1 This is a hardware structure block diagram of the terminal for the generative recommendation semantic tag generation method according to an embodiment of this application;

[0036] Figure 2 This is a flowchart of the generative recommendation semantic tag generation method according to an embodiment of this application;

[0037] Figure 3 This is a flowchart of the item recommendation method according to an embodiment of this application;

[0038] Figure 4 This is a flowchart of a generative recommendation semantic tag generation method according to one embodiment of this application;

[0039] Figure 5 This is a structural block diagram of a generative recommendation semantic tag generation device according to an embodiment of this application. Detailed Implementation

[0040] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0041] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0042] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the generative recommendation semantic tag generation method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0043] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the generative recommendation semantic tag generation method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0044] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0045] In related technologies, semantic ID generation methods mainly include the neural network method of RQ-VAE and the static clustering method of RQ-Kmeans. Among these, the semantic IDs generated by the RQ-VAE method are often of lower quality than those generated by simple static clustering methods. Through analysis and research, the inventors discovered that the main reason for the poor performance of RQ-VAE lies in the severe dimensionality collapse problem caused by the training of the L2 norm loss function. This manifests in two ways: First, the item embeddings input to the RQ-VAE inherently suffer from collapse, thus the degree of dimensionality collapse is even more severe after RQ-VAE reconstruction. Second, both the continuous representation output by the encoder in RQ-VAE and the quantized input representation by the decoder face severe dimensionality collapse. Dimensional collapse refers to the phenomenon where data points in a high-dimensional feature space are actually distributed only in a low-dimensional subspace, resulting in a large number of dimensions failing to carry effective information. Dimensional collapse limits the quality of the final generated semantic IDs.

[0046] More specifically, RQ-VAE extracts latent variable representations z from the input features through an encoder. Then, residual quantization is performed through a multi-layer codebook to obtain the quantized representation. Finally, the original input is reconstructed using a decoder. The RQ-VAE method typically uses the L2 norm loss function (L2 Loss) for neural network training, as shown in the following formula:

[0047] ;

[0048] Where x represents the original input item feature representation; This represents the feature representation of the item after model reconstruction; L represents the total number of residual quantization layers; l represents the l-th quantization layer; This represents the residual representation input to the l-th layer; specifically, These are the continuous latent variables output by the encoder; This represents the codebook representation matched at layer l; This indicates that the gradient operation is stopped, meaning that the variables within this operation do not participate in the gradient calculation during backpropagation; λ and β are hyperparameters that balance the weights of different loss terms.

[0049] After analysis, the inventors discovered that the reasons why the neural network method using RQ-VAE generates semantic IDs suffers from dimensionality collapse and exacerbates information loss are due to the following two aspects:

[0050] One reason is the spectral greedy nature of L2 norm loss. Specifically, because L2 norm loss prioritizes the direction of the largest eigenvalue in the feature space, given that the input item embedding has already collapsed to some extent, the gradient of L2 norm loss will be highly concentrated on the principal spectral components with large singular values ​​(i.e., the larger the eigenvalue, the larger the gradient of the principal spectral component, and the faster the convergence; the smaller the eigenvalue, the smaller the gradient of the secondary spectral component, and the slower the convergence). This causes the neural network to focus on fitting the principal spectral components, completely ignoring the semantic information contained in the secondary spectral components.

[0051] Another aspect is the self-reinforcing collapse loop inherent in the RQ-VAE method. Specifically, during the quantization process of RQ-VAE, the codebook loss and commitment loss, which are part of the loss function, mutually reinforce the collapse effect. These two loss terms bring the input representation and the codebook representation closer together at L2 distance, causing the input and codebook representations of RQ-VAE to exhibit the same trend in dimensionality collapse, forming a vicious cycle that reinforces each other during training.

[0052] To address the aforementioned issues, this embodiment provides a generative recommendation semantic tag generation method. Figure 2 This is a flowchart of the generative recommendation semantic tag generation method in this embodiment, such as... Figure 2 As shown, the process includes the following steps:

[0053] Step S210: Obtain the item representation matrix of the items to be recommended; specifically, the items to be recommended can be item entries on the recommendation system that users can browse, click and purchase; the item representation matrix can be a matrix composed of the number of items to be recommended and the dimension of each item being represented as a vector.

[0054] Step S220: Input the item representation matrix into the trained semantic tag generation model to obtain the item semantic tags generated by the semantic tag generation model; wherein:

[0055] The semantic label generation model is constructed from a neural network based on residual quantization variational autoencoder. The loss function of the semantic label generation model is the loss function after optimizing each L2 norm loss part in the neural network based on residual quantization variational autoencoder using a spectral softening alignment mechanism. The spectral softening alignment mechanism is used to amplify the gradient weights of the minor spectral components of the processed feature values.

[0056] The semantic identifier of an item is a discrete identifier containing item features and semantic information, typically represented as a hierarchical sequence of numbers. RQ-VAE is a neural network model that uses multi-layer codebooks to progressively residual-quantize continuous representations to obtain discrete codes. In this embodiment, during model training, a spectral softening alignment mechanism is constructed to optimize the L2 norm loss of RQ-VAE, forming a new loss function. Model training is then completed based on this new loss function, resulting in a trained semantic identifier generation model. Subsequently, the aforementioned item representation matrix is ​​input into the trained semantic identifier generation model to generate item semantic identifiers for downstream item recommendation. Specifically, after training, the encoder and residual quantization layers with all item features as input parameters are frozen. The codebook indices matching each quantization level are extracted, and the concatenated codebook indices serve as the semantic ID of the item, used in the downstream generative recommendation model.

[0057] This embodiment improves the L2 norm loss of RQ-VAE by employing a strategy of softening and then aligning the spectral signal for each dimension of the spectral domain. This allows for the utilization of secondary spectral information with smaller eigenvalues. Spectral softening refers to softening the intensity of the spectral components of the target representation (obtainable through singular value decomposition) in each loss component of the RQ-VAE L2 norm loss function, resulting in a more uniform weight distribution of the spectral components. Alignment refers to calculating the alignment loss of the softened spectral components of the target representation in each loss component. Therefore, this embodiment optimizes the standard RQ-VAE L2 norm loss function to form a new loss function. Specifically, the reconstruction loss, codebook loss, and commitment loss of the standard RQ-VAE L2 norm loss function can be optimized based on the spectral softening and alignment mechanism to form a new loss function.

[0058] Furthermore, during training, assuming the sample item representation matrix is ​​x, inputting x into the encoder of the RQ-VAE allows for the initial encoding of this item representation matrix, yielding a continuous latent variable representation z. This initial encoding process can be represented as:

[0059] ;

[0060] In the subsequent residual quantization process, z is used as the initial residual. The matching codebook representation is extracted sequentially through multiple codebooks. And calculate the new residuals .

[0061] Understandably, after constructing a new loss function, backpropagation can be performed using this new loss function to update the parameters of the encoder, decoder, and each codebook, thereby training the neural network and obtaining the trained semantic label generation model.

[0062] Therefore, compared with the aforementioned related technologies, there are problems such as the spectral greedy defects caused by L2 norm loss, the tendency to ignore secondary semantic information, and the existence of self-reinforcing collapse loops that lead to dimensional collapse and thus result in low quality of semantic ID generation.

[0063] This embodiment improves the loss function of RQ-VAE from a spectral domain perspective, overcoming the spectral greediness defect of L2 loss. It amplifies the gradient weights of secondary spectral components, thus fully utilizing them and enabling the model to learn the secondary spectral component features of items during optimization, thereby significantly improving the expressive power of semantic IDs. Furthermore, it breaks the self-reinforcing collapse loop of RQ-VAE, alleviating dimensionality collapse and solving the dimensionality collapse problem caused by the loss function. Moreover, it can improve the accuracy of downstream recommendation tasks, breaking through the performance bottleneck of traditional RQ-VAE methods in generative recommendation. Recommendation accuracy can be improved simply by replacing the loss function without changing the model architecture.

[0064] Therefore, through the above steps S210 to S220, the item representation matrix of the item to be recommended is obtained; the item representation matrix is ​​input into the trained semantic tag generation model to obtain the item semantic tag generated by the semantic tag generation model; wherein: the semantic tag generation model is constructed by a neural network based on residual quantization variational autoencoder; the loss function of the semantic tag generation model is the loss function after optimizing each L2 norm loss part in the neural network based on residual quantization variational autoencoder using a spectral softening alignment mechanism; the spectral softening alignment mechanism is used to amplify the gradient weights of the secondary spectral components of the processed feature values. It can amplify the gradient weights of the secondary spectral components of the processed feature values, thereby improving the generation quality of semantic tags.

[0065] In one embodiment, before inputting the item representation matrix into the trained semantic tag generation model to obtain the item semantic tags generated by the semantic tag generation model, the above generative recommendation semantic tag generation method may further include:

[0066] The loss function of a neural network based on a residual quantization variational autoencoder is optimized based on a spectral softening alignment mechanism, and the model is trained to obtain a trained semantic tag generation model. More specifically, in some embodiments, optimizing the loss function of a neural network based on a residual quantization variational autoencoder based on a spectral softening alignment mechanism and training the model to obtain a trained semantic tag generation model may specifically include:

[0067] Singular value decomposition is performed on the target representation matrix of the L2 norm loss part to be optimized, and spectral components are extracted from the results of the singular value decomposition. The softening coefficient of the current training round is determined based on the sigmoid curve function and learnable parameters. The intensity of the spectral components is softened based on the softening coefficient to obtain the target spectral components. The L2 norm loss part is replaced based on the target spectral components to obtain the loss function optimized by the spectral softening alignment mechanism.

[0068] When optimizing the reconstruction loss component of the aforementioned loss function, the target representation matrix is ​​the sample item representation matrix x, and the corresponding prediction matrix participating in the alignment loss calculation is the item feature representation after model reconstruction. When optimizing the codebook loss component of the above loss function, the target representation matrix is ​​the input residual representation of the frozen gradient. The prediction matrix used in the alignment loss calculation is the codebook representation matched by that layer. When optimizing the commitment loss component of the above loss function, the target representation matrix is ​​the codebook representation of the frozen gradient. The prediction matrix used in the alignment loss calculation is a representation of the residuals input to this layer. .

[0069] Assuming the target representation matrix is ​​X, we first perform singular value decomposition on X to extract spectral components. The decomposition formula is as follows:

[0070] ;

[0071] Where, assuming , where N represents the number of items and D represents the dimension of the vector representation of a single item. To extract the spectral components, let the rank of matrix X be r. Let be the left singular value matrix of the target characterization matrix, where each column represents a left spectral vector; The right singular value matrix is ​​the target characterization matrix, where each column represents a right spectral vector; It is a diagonal matrix containing singular values, where each singular value represents the intensity of the corresponding spectral component.

[0072] It should also be noted that, in order to ensure the stability of the training process, in the actual implementation, the gradient of the singular value decomposition process of the target representation matrix X is frozen to avoid instability caused by directly taking the derivative of the singular value decomposition process.

[0073] Then, the diagonal matrix after the singular value decomposition can be... The eigenvalues ​​in the spectrum are softened to compress the relative weights of the primary eigenvalues ​​and increase the relative weights of the secondary eigenvalues. Specifically, the calculation method used for this spectral softening operation can be:

[0074] ;

[0075] in, The singular value diagonal matrix represents the spectrum after softening; α is the softening coefficient; specifically, in order for the neural network in this embodiment to adaptively learn the degree of softening, the calculation process of α can be set as follows:

[0076] ;

[0077] Here, 'a' is a learnable parameter. During the initial training phase, parameter 'a' can be initialized to 0. Afterward, the value of 'a' is automatically adjusted as the neural network trains. Specifically, based on the training objective of minimizing the loss function, the update direction of parameter 'a' is calculated using gradient descent, and the learning rate is used as the parameter update step size, automatically updating it during each training iteration.

[0078] Then, the softened diagonal matrix is ​​used. The softened target representation matrix is ​​reconstructed, and the alignment loss is calculated between it and the prediction matrix. This replaces the L2 norm loss components. Assume the original alignment loss is: Then, the alignment loss improved based on the softened diagonal matrix is: .in, The prediction matrix corresponding to the target characterization matrix is ​​used to calculate the alignment loss.

[0079] Here, before softening, the diagonal matrix is... The expanded form is denoted as:

[0080] ;

[0081] in, Represents the diagonal elements of a diagonal matrix. This represents the weight of each spectral component. The original spectral signal suffers from spectral collapse, which manifests as an uneven intensity distribution (weight distribution) among different spectral components. The first few spectral components almost occupy all the spectral information, resulting in a larger number of secondary spectral components being ignored. Therefore, this embodiment softens the spectral signal, essentially making the weight distribution of the spectral signal more uniform, thereby alleviating this spectral collapse problem.

[0082] Therefore, this embodiment extracts spectral components based on singular value decomposition, uses a softening coefficient to soften the intensity of the spectral components, and then optimizes the loss function, which can overcome the dimensionality collapse problem and improve the generation quality of semantic IDs.

[0083] This embodiment applies the spectral softening alignment mechanism to the three core L2 norm loss components of RQ-VAE, specifically to the reconstruction loss, codebook loss, and commitment loss.

[0084] Specifically, in one embodiment, the L2 norm loss component is replaced based on the target spectral components to obtain a loss function optimized by the spectral softening alignment mechanism, which may specifically include:

[0085] When the L2 norm loss to be optimized is the reconstruction loss, the reconstruction loss is replaced with: the alignment loss between the target spectral component of the sample item representation matrix and the predicted representation matrix; the predicted representation matrix is ​​the item representation matrix predicted by the neural network based on the residual quantization variational autoencoder in the current training round.

[0086] When the spectral softening alignment mechanism is applied to the reconstruction loss term, the target representation matrix is ​​the original sample item representation matrix x, which does not participate in gradient updates. The alignment loss to be calculated is the sample item representation matrix x and the item representation matrix after model reconstruction. The final mathematical expression for the reconstruction loss is represented by the sample item representation matrix x and the reconstructed item representation matrix x. Spectral softening alignment loss between :

[0087] ;

[0088] Therefore, this embodiment can achieve spectral softening alignment optimization for reconstruction loss.

[0089] Specifically, in one embodiment, the L2 norm loss component is replaced based on the target spectral components to obtain a loss function optimized by the spectral softening alignment mechanism, which may specifically include:

[0090] When the L2 norm loss to be optimized is the codebook loss, the codebook loss is replaced with the alignment loss between the target spectral component of the input residual representation and the codebook representation.

[0091] When the spectral softening alignment mechanism is applied to the codebook loss term, for the l-th quantization layer, the target representation matrix is ​​set as the input residual representation of the frozen gradient. Its singular value decomposition is represented as The prediction matrix used in calculating the alignment loss is the codebook representation matched by that layer. At this point, the object of singular value decomposition is directly without gradient. Since the target representation matrix of codebook loss naturally contains The operation, therefore, naturally implements gradient freezing in the corresponding singular value decomposition process. Thus, codebook loss based on spectral softening alignment... The specific expression can be:

[0092] ;

[0093] Therefore, this embodiment can achieve spectral softening alignment optimization of the codebook loss portion.

[0094] Specifically, in one embodiment, the L2 norm loss component is replaced based on the target spectral components to obtain a loss function optimized by the spectral softening alignment mechanism, which may specifically include:

[0095] When the L2 norm loss to be optimized is the commitment loss, the commitment loss is replaced with the alignment loss between the target spectral component represented by the codebook and the residual representation.

[0096] Specifically, when the spectral softening alignment mechanism is applied to the commitment loss part, the target representation matrix is ​​set as the codebook representation of the frozen gradient. Its singular value decomposition is represented as Since the target representation matrix is ​​a codebook representation with a stopping gradient operation. At this point, the object of singular value decomposition is directly without gradient. Because the target representation matrix in the commitment loss naturally contains The operation, therefore, naturally implements gradient freezing in the corresponding singular value decomposition process. The prediction matrix involved in the alignment loss calculation is the residual representation input to this l-layer. The specific mathematical expression for the committed loss is the summation over L levels, expressed as:

[0097] ;

[0098] Therefore, this embodiment achieves spectral softening alignment optimization for the commitment loss portion.

[0099] Therefore, by combining the above three parts and using hyperparameters... and By balancing the weights of the different loss terms, the overall loss function after the replacement is expressed as:

[0100] ;

[0101] in, For the reconstruction loss term after spectral softening and alignment optimization, For codebook loss term after spectral softening and alignment optimization; The commitment loss term is optimized for spectral softening and alignment.

[0102] That is, it is:

[0103] ;

[0104] This embodiment focuses on revealing and solving the dimensionality collapse problem in semantic ID generation using RQ-VAE from the perspective of spectral analysis. A loss function calculation mechanism based on spectral softening alignment is constructed: by performing singular value decomposition on the target representation matrix, a learnable power-law softening operation is used to smooth the feature matrix, reconstructing the softened target, and calculating the distance between the model output and the softened target (alignment loss). Furthermore, this mechanism systematically replaces the traditional L2 loss used for reconstruction, codebook update, and commitment in RQ-VAE, forming a complete training framework. Thus, this embodiment breaks the self-reinforcing collapse cycle and alleviates the dimensionality collapse problem. Specifically, regarding the representation homogenization and dimensionality collapse caused by the codebook loss and commitment loss converging, this embodiment extends the application of spectral softening alignment to the quantization loss, preventing codebook update and feature alignment from blindly fitting the dominant dimension, thereby ensuring the information entropy and diversity of the generated semantic ID across all dimensions.

[0105] This embodiment also provides a method for recommending items. Figure 3 This is a flowchart of the item recommendation method in this embodiment, such as... Figure 3 As shown, the process includes the following steps:

[0106] Step S301: Execute the generative recommendation semantic identifier generation method provided in any of the above embodiments to determine the item semantic identifier of the item to be recommended;

[0107] Step S302: Based on the semantic identifiers of items and the user's historical behavior information, combined with the trained generative recommendation model, the item recommendation results are obtained.

[0108] The user's historical behavior information can include information about items the user has clicked, browsed, or purchased in the past. After generating the aforementioned item semantic identifiers, the user's historical interaction sequence (i.e., the items the user has clicked in the past) can be converted into a corresponding item semantic identifier sequence (semantic ID sequence), which is then input into the generative recommendation model. The generative recommendation model learns the patterns in these sequences and attempts to predict the semantic ID of the next most likely item to be interacted with. Finally, the generative recommendation model outputs the semantic ID of the next item, completing the item recommendation.

[0109] Through the above steps S301 to S302, by optimizing the loss function based on the spectral softening alignment mechanism in the semantic ID generation stage, the generation quality of semantic IDs can be improved, thereby optimizing the downstream item recommendation performance.

[0110] Figure 4This is a flowchart of a generative recommendation semantic tag generation method according to one embodiment, such as... Figure 4 As shown, the generative recommendation semantic tag generation method includes the following steps:

[0111] Step S401: During the model training phase, the sample item representation matrix is ​​input and preliminarily encoded; wherein, the encoder based on RQ-VAE performs the preliminary encoding to obtain continuous latent variable representations.

[0112] Step S402: Perform singular value decomposition on the target representation matrix to obtain the spectral components; the target representation matrix varies depending on the different loss components optimized, as described in the above embodiments, and will not be repeated here.

[0113] Step S403 involves performing learnable softening on the spectral signal based on the softening coefficient. Specifically, the learnable parameters are input into the S-curve function to obtain the softening coefficient, and the intensity of the spectral components is enhanced using the softening coefficient. The specifics are as described in the above embodiments and will not be repeated here.

[0114] Step S404: Based on the spectral softening alignment mechanism, perform spectral softening alignment optimization on each loss component of the loss function, and calculate the spectral softening alignment loss of each component.

[0115] Step S405: Construct a global loss function and complete the training to obtain the trained semantic label generation model.

[0116] Step S406: Generate item identifiers using the trained semantic identifier generation model.

[0117] Steps S401 to S406 above improve the loss function of RQ-VAE from a spectral domain perspective, overcoming the spectral greediness defect of L2 loss. They amplify the gradient weights of secondary spectral components, fully utilizing them and enabling the model to learn the secondary spectral component features of items during optimization, thus significantly improving the expressive power of semantic IDs. Furthermore, they break the self-reinforcing collapse loop of RQ-VAE, alleviating dimensional collapse and solving the dimensional collapse problem caused by the loss function. Moreover, they improve the accuracy of downstream recommendation tasks, overcoming the performance bottleneck of traditional RQ-VAE methods in generative recommendation. Recommendation accuracy can be improved simply by replacing the loss function without changing the model architecture.

[0118] The following describes the experimental verification results of the generative recommendation semantic tag generation method of this embodiment on two categories (Beauty and Toys) in the Amazon public recommendation dataset. It should be noted that the experimental verification process was carried out with the same dataset processing method as existing related technologies, and grid search was used to find the optimal combination of hyperparameters such as learning rate and weight decay factor. The evaluation metrics used are the hit rate (H@K) and normalized loss cumulative gain (N@K), which are widely used in recommendation systems.

[0119] In the first experimental example, the main focus is on verifying the performance comparison between the generative recommendation semantic tag generation method of this embodiment and the traditional generative recommendation semantic tag generation method under the same experimental settings. The experiment uses the classic generative recommendation method based on the standard RQ-VAE (“Rajput S, Mehta N, Singh A, et al. Recommendmender systems with generative retrieval[J]. Advances in Neural Information Processing Systems, 2023, 36: 10299-10315”, abbreviated as “TIGER”) as the baseline method, and introduces the spectral softening alignment loss function proposed in this embodiment. The experimental results are shown in Table 1:

[0120] Table 1

[0121]

[0122] The experimental results in Table 1 demonstrate that by introducing the spectral softening alignment optimization loss function of this embodiment, the quality of the generated item semantic labels and the recommendation accuracy of the downstream model are significantly improved. Both H@Ky and N@K metrics are significantly enhanced, with H@5 reaching a maximum improvement of 0.0518 on the Beauty dataset (a 7.25% improvement compared to the baseline method), surpassing the performance of the baseline method. An average improvement of 3.94% was achieved across all metrics on the Beauty and Toys datasets.

[0123] In the second experimental example, to verify the effectiveness of the generative recommendation semantic tag generation method proposed in this embodiment in alleviating the dimensionality collapse of model representations, a comparative experiment was conducted between the method provided in this embodiment and the classic generative recommendation method ("TIGER") based on the standard RQ-VAE, examining the dimensionality collapse on various representations. This experiment was conducted on the Amazon Beauty dataset and evaluated using the percentage of the cumulative sum of the top 10% of singular values, which measures the degree of dimensionality collapse (a smaller value indicates a lower degree of dimensionality collapse). The experimental results are shown in Table 2.

[0124] Table 2

[0125]

[0126] The experimental results in Table 2 show that the method in this embodiment significantly alleviates the dimensionality collapse problem faced by the existing RQ-VAE method. This embodiment can simultaneously alleviate the degree to which the singular values ​​of the reconstructed representation, the pre-quantization representation, and the post-quantization representation are dominated by the main spectral components, thereby improving the quality and expressive power of the representation and ultimately generating higher quality semantic labels.

[0127] This embodiment also provides a generative recommendation semantic tag generation device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated for details already described. The terms "module," "unit," "subunit," etc., used below can refer to combinations of software and / or hardware that implement a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0128] Figure 5 This is a structural block diagram of the generative recommendation semantic tag generation device 50 in this embodiment, as shown below. Figure 5 As shown, the generative recommendation semantic tagging generation device 50 includes: an acquisition module 52 and a generation module 54; wherein:

[0129] The acquisition module 52 is used to acquire the item representation matrix of the item to be recommended; the generation module 54 is used to input the item representation matrix into the trained semantic tag generation model to obtain the item semantic tag generated by the semantic tag generation model; wherein: the semantic tag generation model is constructed by a neural network based on residual quantization variational autoencoder; the loss function of the semantic tag generation model is the loss function after optimizing each L2 norm loss part in the neural network based on residual quantization variational autoencoder by a spectral softening alignment mechanism; the spectral softening alignment mechanism is used to amplify the gradient weights of the secondary spectral components of the processed feature values.

[0130] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0131] In one embodiment, the generative recommendation semantic tag generation device 50 further includes a training module, which is used to optimize the loss function of the neural network based on the residual quantization variational autoencoder based on the spectral softening alignment mechanism, and to train the model to obtain the trained semantic tag generation model.

[0132] In one embodiment, the training module is specifically used to perform singular value decomposition on the target representation matrix of the L2 norm loss part to be optimized, and extract spectral components from the results of the singular value decomposition; determine the softening coefficient of the current training round based on the sigmoid curve function and learnable parameters; soften the intensity of the spectral components based on the softening coefficient to obtain the target spectral components; and replace the L2 norm loss part based on the target spectral components to obtain the loss function optimized by the spectral softening alignment mechanism.

[0133] In one embodiment, the training module is specifically used to replace the reconstruction loss part with the alignment loss between the target spectral component of the sample item representation matrix and the predicted representation matrix when the L2 norm loss part to be optimized is the reconstruction loss part; the predicted representation matrix is ​​the item representation matrix predicted by the neural network based on the residual quantization variational autoencoder in the current training round.

[0134] In one embodiment, the training module is specifically used to replace the codebook loss with the alignment loss between the target spectral component of the input residual representation and the codebook representation when the L2 norm loss to be optimized is the codebook loss.

[0135] In one embodiment, the training module is specifically used to replace the commitment loss part with the alignment loss between the target spectral component of the codebook representation and the residual representation when the L2 norm loss part to be optimized is the commitment loss part.

[0136] This embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0137] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0138] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0139] S1, Obtain the item representation matrix of the items to be recommended;

[0140] S2, input the item representation matrix into the trained semantic labeling generation model to obtain the item semantic labels generated by the semantic labeling generation model; where:

[0141] The semantic label generation model is constructed from a neural network based on residual quantization variational autoencoder. The loss function of the semantic label generation model is the loss function after optimizing each L2 norm loss part in the neural network based on residual quantization variational autoencoder using a spectral softening alignment mechanism. The spectral softening alignment mechanism is used to amplify the gradient weights of the minor spectral components of the processed feature values.

[0142] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0143] Furthermore, in conjunction with the generative recommendation semantic tag generation method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the generative recommendation semantic tag generation methods in the above embodiments.

[0144] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0146] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0147] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A generative recommendation semantic tag generation method, characterized in that, include: Obtain the item representation matrix of the items to be recommended; The item representation matrix is ​​input into the trained semantic identifier generation model to obtain the item semantic identifier generated by the semantic identifier generation model; wherein: The semantic identifier generation model is constructed from a neural network based on residual quantization variational autoencoder; the loss function of the semantic identifier generation model is the loss function after optimizing each L2 norm loss part in the neural network based on residual quantization variational autoencoder using a spectral softening alignment mechanism; the spectral softening alignment mechanism is used to amplify the gradient weights of the secondary spectral components of the processed feature values.

2. The generative recommendation semantic tag generation method according to claim 1, characterized in that, Before inputting the item representation matrix into the trained semantic tag generation model to obtain the item semantic tags generated by the semantic tag generation model, the method further includes: The loss function of the neural network based on residual quantization variational autoencoder is optimized based on the spectral softening alignment mechanism, and the model is trained to obtain the trained semantic label generation model.

3. The generative recommendation semantic tag generation method according to claim 2, characterized in that, The loss function of the neural network based on residual quantization variational autoencoder is optimized based on the spectral softening alignment mechanism, and the model is trained to obtain the trained semantic label generation model, including: Singular value decomposition is performed on the target representation matrix of the L2 norm loss part to be optimized, and spectral components are extracted from the results of the singular value decomposition. The softening coefficient for the current training round is determined based on the S-curve function and learnable parameters. The intensity of the spectral component is softened based on the softening coefficient to obtain the target spectral component; The L2 norm loss component is replaced based on the target spectral components to obtain the loss function optimized by the spectral softening alignment mechanism.

4. The generative recommendation semantic tag generation method according to claim 3, characterized in that, The L2 norm loss component is replaced based on the target spectral components to obtain the loss function optimized by the spectral softening alignment mechanism, including: When the L2 norm loss to be optimized is the reconstruction loss, the reconstruction loss is replaced with the alignment loss between the target spectral component of the sample item representation matrix and the predicted representation matrix; the predicted representation matrix is ​​the item representation matrix predicted by the neural network based on the residual quantization variational autoencoder in the current training round.

5. The generative recommendation semantic tag generation method according to claim 3, characterized in that, The L2 norm loss component is replaced based on the target spectral components to obtain the loss function optimized by the spectral softening alignment mechanism, including: When the L2 norm loss to be optimized is the codebook loss, the codebook loss is replaced with the alignment loss between the target spectral component of the input residual representation and the codebook representation.

6. The generative recommendation semantic tag generation method according to claim 3, characterized in that, The L2 norm loss component is replaced based on the target spectral components to obtain the loss function optimized by the spectral softening alignment mechanism, including: When the L2 norm loss to be optimized is the commitment loss, the commitment loss is replaced with the alignment loss between the target spectral component represented by the codebook and the residual representation.

7. A method for recommending items, characterized in that, The method includes: The generative recommendation semantic identifier generation method according to any one of claims 1 to 6 is used to determine the item semantic identifier of the item to be recommended; Based on the semantic identifiers of the items and the user's historical behavior information, combined with the trained generative recommendation model, the item recommendation results are obtained.

8. A generative recommendation semantic tag generation device, characterized in that, include: The module for obtaining data and the module for generating data; where: The acquisition module is used to acquire the item representation matrix of the item to be recommended; The generation module is used to input the item representation matrix into the trained semantic identifier generation model to obtain the item semantic identifier generated by the semantic identifier generation model; wherein: The semantic identifier generation model is constructed from a neural network based on residual quantization variational autoencoder; the loss function of the semantic identifier generation model is the loss function after optimizing each L2 norm loss part in the neural network based on residual quantization variational autoencoder using a spectral softening alignment mechanism; the spectral softening alignment mechanism is used to amplify the gradient weights of the secondary spectral components of the processed feature values.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the generative recommendation semantic identifier generation method according to any one of claims 1 to 6, or to perform the item recommendation method according to claim 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the generative recommendation semantic identifier generation method according to any one of claims 1 to 6, or the steps of the item recommendation method according to claim 7.

Images