An article recommendation method, device, storage medium and electronic device

Abstract

The embodiment of the specification discloses an article recommendation method, device, medium and equipment, the method comprises: in the article recommendation scene, after determining the original article feature vector of multiple articles under multiple modalities is mapped to a unified space, the initial article feature vector is obtained; the initial article feature vector is subjected to first frequency domain transformation to obtain initial frequency domain representation; the target frequency domain representation is obtained by adjusting the phase distribution offset of the initial frequency domain representation, and the second frequency domain transformation is performed to obtain the target article feature vector; training a multi-modal recommendation model according to the target article feature vector, and recommending articles to users based on the multi-modal recommendation model. By using the above technical scheme, the specific noise in the modal is suppressed by performing frequency domain transformation and phase offset adjustment on the feature vector of multiple modalities mapped to the unified space, the phase spectrum offset between modalities is calibrated, and the shared preference between modalities is enhanced. The semantic consistency of the recommendation system is improved, and the accuracy of article recommendation is improved.

Description

Technical Field

[0001] This specification relates to the field of product recommendation technology, and in particular to a product recommendation method, apparatus, storage medium, and electronic device. Background Technology

[0002] With the development of recommender system technology, traditional methods based solely on user-item interaction behavior are no longer sufficient to fully characterize the item's attributes and the user's fine-grained interests and preferences. To improve recommendation accuracy, academia and industry have gradually incorporated multimodal information such as images, text, and audio into recommender systems, forming a multimodal recommendation technology approach. However, existing multimodal recommendation methods often ignore the specific noise differences between modalities, easily introducing irrelevant perturbations, and lack intermodal phase spectrum shift calibration and shared information, leading to distorted fusion representations and semantic inconsistencies, thus affecting the accuracy of recommendations. Summary of the Invention

[0003] This specification provides an item recommendation method, apparatus, storage medium, and electronic device, the technical solutions of which are as follows: Firstly, embodiments of this specification provide a method for recommending items, the method comprising: In the item recommendation scenario, the original item feature vectors of multiple items in at least two modalities are determined, and the original item feature vectors are mapped to a unified representation space to obtain the initial item feature vectors of the multiple items in each different modality. Perform a first frequency domain transformation on each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode; In the frequency domain, the initial frequency domain representation of each mode is phase-distribution offset adjusted to obtain the target frequency domain representation, and the target frequency domain representation is subjected to a second frequency domain transformation to obtain the target item feature vector under each different mode; A multimodal recommendation model is trained based on the feature vector of the target item under each different modality, and the user is recommended items based on the multimodal recommendation model.

[0004] In one feasible implementation, the first frequency domain transformation of each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode includes: The initial item feature vectors are transformed from the time domain or spatial domain to the frequency domain to obtain the initial frequency domain representations at multiple frequency points under each different mode.

[0005] In one feasible implementation, the step of adjusting the phase distribution offset of the initial frequency domain representation of each mode in the frequency domain to obtain the target frequency domain representation includes: Based on the phase spectrum vector of the multiple items in each different mode, the phase offset of the multiple items at the corresponding multiple frequency points is determined, and the initial frequency domain characterization is calibrated based on the phase offset to obtain the first intermediate frequency domain characterization. The filtering weights of the multiple items at the corresponding multiple frequency points are determined based on the amplitude of the first intermediate frequency domain representation, and the amplitude of the first intermediate frequency domain representation is adjusted based on the filtering weights to obtain the second intermediate frequency domain representation. The amplitude squared coherence coefficient is determined based on the first intermediate frequency domain representation of the multiple items in each different mode and the amplitude of the first intermediate frequency domain representation. The first intermediate frequency domain representation is then reweighted based on the amplitude squared coherence coefficient to obtain the third intermediate frequency domain representation. The target frequency domain representation is obtained by weighted fusion of the second intermediate frequency domain representation and the third intermediate frequency domain representation.

[0006] In one feasible implementation, the at least two modalities include a text modality and an image modality. The step of determining the phase shift of the plurality of items at corresponding frequency points based on the phase spectral vectors of the plurality of items in each different modality, and calibrating the initial frequency domain representation based on the phase shift to obtain a first intermediate frequency domain representation, includes: The phase spectrum vectors of the multiple items in text mode and image mode are obtained. Based on the phase spectrum vectors, the first phase offset of the multiple items at multiple frequency points is determined using a first deviation calculation formula. Based on the first phase offset, the second phase offset is determined using a second deviation calculation formula. According to the second phase offset, the initial frequency domain representations of the text mode and image mode are symmetrically phase rotated and calibrated. The residual deviation of the initial frequency domain representation after symmetrical phase rotation calibration is compensated using a calibration calculation formula to obtain the first intermediate frequency domain representation. The first deviation calculation formula satisfies the following formula:

[0007] in, Let i be the first phase offset of the i-th item. Let be the phase spectrum vector of the i-th item in the text modality, and . Let be the phase spectrum vector of the i-th item in the image mode; The second deviation calculation formula satisfies the following formula:

[0008] in, This is the second phase offset; The calibration calculation formula satisfies the following formula:

[0009] in, For the initial frequency domain representation of the i-th item in the image modality, This represents the initial frequency domain representation of the i-th item in the text modality. This is the residual phase compensation vector. This represents the first intermediate frequency domain of the i-th item in the image modality. This represents the first intermediate frequency domain of the i-th item in the text modality.

[0010] In one feasible implementation, determining the filtering weights of the plurality of items at corresponding frequency points based on the amplitude of the first intermediate frequency domain representation, and adjusting the amplitude of the first intermediate frequency domain representation based on the filtering weights to obtain the second intermediate frequency domain representation, includes: The amplitude of the multiple items represented in the first intermediate frequency domain at the same frequency point in the same mode is determined. The median absolute deviation is calculated using the median absolute deviation calculation formula based on the amplitude. The noise variance is determined using the noise variance calculation formula based on the median absolute deviation. The total amplitude variance of the multiple items at the same frequency point is determined. The effective semantic variance is determined using the effective semantic variance calculation formula based on the effective semantic variance. The filter weight is determined using the filter weight calculation formula based on the filter weight. The amplitude represented in the first intermediate frequency domain is adjusted using the amplitude adjustment calculation formula based on the filter weight to obtain the second intermediate frequency domain representation. The formula for calculating the absolute deviation of the median satisfies the following formula:

[0011]

[0012] Where m represents the text modality or the image modality, This represents the median absolute deviation of the text or image modality at the f-th frequency point. The amplitude of the i-th item at the f-th frequency point in either text or image mode after symmetric phase rotation calibration; The noise variance calculation formula satisfies the following formula:

[0013] in, For text modality or image modality The noise variance at the f-th frequency point This is a constant used to correct for the absolute deviation of the median; The effective semantic variance calculation formula satisfies the following formula:

[0014] in, For text modality or image modality Effective semantic variance at the f-th frequency point For text modality or image modality The total variance of the amplitude at the f-th frequency point; The formula for calculating the filter weights satisfies the following:

[0015] in, For text modality or image modality The filter weights at the f-th frequency point; The amplitude adjustment calculation formula satisfies the following formula:

[0016] in, It represents the second intermediate frequency domain of the i-th item at the f-th frequency point in either the text or image modality.

[0017] In one feasible implementation, the step of determining the amplitude squared coherence coefficient based on the first intermediate frequency domain representation of the plurality of items in each different mode and the amplitude of the first intermediate frequency domain representation, and reweighting the first intermediate frequency domain representation based on the amplitude squared coherence coefficient to obtain the third intermediate frequency domain representation, includes: Based on the first intermediate frequency domain characterization of the multiple items in each different mode, the cross-modal cross power spectrum of the items at multiple frequency points is determined by the cross power spectrum calculation formula. Based on the amplitude of the first intermediate frequency domain characterization, the rate spectral density of the items in each different mode is determined by the power spectral density calculation formula. Based on the cross-modal cross power spectrum and the rate spectral density, the amplitude squared coherence coefficient of the items at multiple frequency points is determined by the amplitude squared coherence coefficient calculation formula. Based on the amplitude squared coherence coefficient, the first intermediate frequency domain characterization is reweighted by the reweighting calculation formula to obtain the third intermediate frequency domain characterization. The cross-power spectrum calculation formula satisfies the following formula:

[0018] in, Let be the cross-modal power spectrum of the i-th item at multiple frequency points, denoted by . For element-wise multiplication, For complex conjugate; The power spectral density calculation formula satisfies the following formula:

[0019] in, Let be the power spectral density of the i-th item in the text mode or image mode; The formula for calculating the amplitude squared coherence coefficient satisfies the following formula:

[0020] in, Let be the amplitude squared coherence coefficient of the i-th item at multiple frequency points. For the i-th item Power spectral density in text mode, For the i-th item Power spectral density in image mode; The reweighted calculation formula satisfies the following formula:

[0021] in, It represents the third intermediate frequency domain of the i-th item at multiple frequency points in the text or image modality.

[0022] In one feasible implementation, the at least two modalities include a text modality and an image modality, and the weighted fusion of the second intermediate frequency domain representation and the third intermediate frequency domain representation to obtain the target frequency domain representation includes: Determine the fusion weights, and use a weighted fusion calculation formula based on the fusion weights to perform weighted fusion of the second intermediate frequency domain representation and the third intermediate frequency domain representation to obtain the target frequency domain representation. The weighted fusion calculation formula satisfies the following formula:

[0023] in, The fusion weights are represented by the second intermediate frequency domain. The fusion weights represented by the third intermediate frequency domain are 1, and the sum of the fusion weights represented by the second and third intermediate frequency domains is 1. This represents the target frequency domain of the i-th item in either the text or image modal.

[0024] In one feasible implementation, the step of performing a second frequency domain transformation on the target frequency domain representation to obtain the target item feature vectors under different modes includes: The target frequency domain representation is transformed from the frequency domain to the time domain or the spatial domain to obtain the target item feature vector under each different mode.

[0025] In one feasible implementation, training a multimodal recommendation model based on the target item feature vectors under different modalities, and performing item recommendation processing on the user based on the multimodal recommendation model, includes: Obtain the user's historical interaction information; The feature vectors of the target items under each different modality are input into the multimodal recommendation model; The user's historical interaction information is used as a supervision signal to train the multimodal recommendation model; The trained multimodal recommendation model is used to recommend items to users.

[0026] In one feasible implementation, the method of recommending items to a user based on the trained multimodal recommendation model includes: The user item feature vector of the target user and the target item feature vectors of multiple items are input into the trained multimodal recommendation model; The trained multimodal recommendation model predicts the probability of interaction between the target user and multiple items. Based on the interaction probability, item recommendation processing is performed on the target user for the multiple items.

[0027] Secondly, embodiments of this specification provide an item recommendation device, the device comprising: The determination module is used to determine the original item feature vectors of multiple items in at least two modalities in an item recommendation scenario, and to map the original item feature vectors to a unified representation space to obtain the initial item feature vectors of the multiple items in each different modality. The transformation module is used to perform a first frequency domain transformation on each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode. The adjustment module is used to perform phase distribution offset adjustment on the initial frequency domain representation of each mode in the frequency domain to obtain the target frequency domain representation, and to perform a second frequency domain transformation on the target frequency domain representation to obtain the target item feature vector under each different mode; The recommendation module is used to train a multimodal recommendation model based on the feature vector of the target item under each different modality, and to perform item recommendation processing for users based on the multimodal recommendation model.

[0028] Thirdly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0029] Fourthly, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0030] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following: In one or more embodiments of this specification, in an item recommendation scenario, after determining the original item feature vectors of multiple items under various modalities, they are mapped to a unified space to obtain initial item feature vectors. A first frequency domain transformation is performed on the initial item feature vectors to obtain an initial frequency domain representation. A phase distribution offset adjustment is applied to the initial frequency domain representation to obtain a target frequency domain representation, and a second frequency domain transformation is performed to obtain a target item feature vector. A multimodal recommendation model is trained based on the target item feature vector, and items are recommended to the user based on the multimodal recommendation model. By employing the above technical solution, frequency domain transformation and phase offset adjustment are performed on the feature vectors of multiple modalities mapped to a unified space to suppress specific noise within a modality, calibrate the phase spectrum offset between modalities, and enhance shared preferences between modalities. This improves the semantic consistency of the recommendation system and enhances the accuracy of item recommendations. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart illustrating an item recommendation method provided in the embodiments of this specification; Figure 2 This is a flowchart illustrating another item recommendation method provided in the embodiments of this specification; Figure 3 This is a schematic diagram of a phase distribution offset adjustment process provided in the embodiments of this specification; Figure 4 This is a flowchart illustrating another item recommendation method provided in the embodiments of this specification; Figure 5 This is a schematic diagram of the structure of an item recommendation device provided in the embodiments of this specification; Figure 6 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification; Figure 7 This is a schematic diagram of the operating system and user space structure provided in the embodiments of this specification; Figure 8 yes Figure 7 Architecture diagram of the Android operating system in China; Figure 9 yes Figure 7 Architecture diagram of the iOS operating system. Detailed Implementation

[0033] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0034] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0035] The present specification will now be described in detail with reference to specific embodiments.

[0036] In one embodiment, such as Figure 1 As shown, a method for recommending items is proposed. This method can be implemented using a computer program and can run on an item recommendation device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application. The item recommendation device can be a terminal device, including but not limited to: personal computers, tablets, handheld devices, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem. In different networks, the terminal device can be called by different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user equipment, cellular phone, cordless phone, terminal device in 5G networks or future evolved networks, etc.

[0037] Specifically, the recommended methods for using this item include: S102: In the item recommendation scenario, determine the original item feature vectors of multiple items in at least two modalities, and map the original item feature vectors to a unified representation space to obtain the initial item feature vectors of the multiple items in each different modality; The original item feature vector can be understood as an unprocessed, low-level numerical representation extracted directly from different modalities of data such as images and text, including background, occlusion, irrelevant regions, and other noise.

[0038] A unified representation space can be understood as a vector space in which original features of different dimensions are projected onto a common target dimension through a linear mapping matrix. It is used to eliminate the differences in original dimensions between different modalities such as images and text.

[0039] The initial item feature vector can be understood as a standardized vector representation of the original features after dimensionality unification mapping, which is within the same semantic scale and numerical range.

[0040] For example, in a multimodal item recommendation scenario, data from different modalities such as images, text, and audio are originally in completely different dimensional spaces. They must be transformed into a unified representation space of the same dimension through mapping so that the feature vectors of different modalities can meet the dimensional consistency of mathematical operations, which is convenient for subsequent mathematical operations.

[0041] For example, the i-th item in the modal The original item feature vector Mapping to a unified representation space yields the initial item feature vector. The corresponding calculation method is as follows:

[0042] in, and Modal The corresponding learnable parameters. For the i-th item in mode The initial item feature vector.

[0043] S104: Perform a first frequency domain transformation on each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode; The first frequency domain transformation, which can be the Fast Fourier Transform, is used to transform a vector-form signal from its original time or spatial domain representation to its frequency domain representation, thereby revealing the distribution characteristics of the signal at different frequency components.

[0044] The initial frequency domain representation can be understood as the complex number representation obtained after the first frequency domain transformation. The value at each frequency point contains a real part and an imaginary part, which can be further decomposed into amplitude spectrum components (amplitude) and phase spectrum components. The amplitude spectrum reflects the energy intensity of the signal at each frequency, and the phase spectrum reflects the relative positional relationship of each frequency component.

[0045] For example, the amplitude spectrum reflects which frequency components dominate in a mode; while the phase spectrum carries the alignment and structural information between different frequency components, determining the specific form of the signal in the spatial domain. Through this frequency domain decomposition, the amplitude and phase information, which were originally coupled in the spatial domain, are successfully separated, providing operational space for subsequent calculations.

[0046]

[0047] in, This represents the amplitude spectral components of the initial frequency domain characterization. This represents the phase spectrum component characterized in the initial frequency domain.

[0048] S106: In the frequency domain, the initial frequency domain representation of each mode is adjusted by phase distribution offset to obtain the target frequency domain representation, and the target frequency domain representation is transformed by a second frequency domain to obtain the target item feature vector under each different mode; The target frequency domain representation can be understood as the final frequency domain representation obtained after the initial frequency domain representation has undergone phase distribution offset adjustment. The target frequency domain representation is superior to the initial frequency domain representation in terms of phase consistency, noise suppression, and enhanced shared information. The phase distribution offset adjustment can include calibrating the phase spectrum shift between modes, suppressing specific noise within modes, and enhancing shared preferences between modes.

[0049] The second frequency domain transformation can be the inverse fast Fourier transform, used to reconstruct the complex frequency domain representation after frequency domain processing back to the time domain or spatial domain, to obtain the target item feature vector that can be directly used by the multimodal recommendation model.

[0050] For example, in the frequency domain, phase information determines the shape and structure of a signal. Semantic misalignments often exist between different modes, which are typically manifested as phase shifts. Correcting these misalignments by adjusting the phase distribution shift aligns the different modes in the frequency domain. Further adjustments to the frequency domain representation suppress intramodal noise, calibrate phase spectrum shifts between modes, and enhance shared preferences between modes are then performed. After these adjustments, a second frequency domain transformation returns the signal to the vector space, resulting in a more accurate target item feature vector than the initial feature vector.

[0051] S108: Train a multimodal recommendation model based on the feature vector of the target item under each different modality, and perform item recommendation processing on the user based on the multimodal recommendation model.

[0052] A multimodal recommendation model can be understood as a recommendation model with a graph neural network or attention mechanism as its backbone network. When training it, the input includes the user's historical interaction behavior features and the feature vector of the target item, and the output can be the target item with high user preference value.

[0053] For example, the user's historical interaction behavior features and the target item feature vector are jointly input into the recommendation model. The user's actual historical clicks or purchases serve as supervision signals, and the model parameters are optimized using a backpropagation algorithm. Since the input target feature vector has undergone frequency domain phase calibration, intra-modal noise suppression, and cross-modal information sharing enhancement, the model can obtain signals that are more focused on shared semantics when learning user preferences, thereby improving training efficiency and convergence. During the inference phase, the trained model is deployed online to perform batch rating predictions for the target user and items in the candidate item library. Based on the predicted ratings, a personalized recommendation list is generated and returned to the user.

[0054] In the embodiments of this specification, in an item recommendation scenario, after determining the original item feature vectors of multiple items under various modalities, they are mapped to a unified space to obtain initial item feature vectors. A first frequency domain transformation is performed on the initial item feature vectors to obtain an initial frequency domain representation. A phase distribution offset adjustment is applied to the initial frequency domain representation to obtain a target frequency domain representation, and a second frequency domain transformation is performed to obtain the target item feature vector. A multimodal recommendation model is trained based on the target item feature vector, and items are recommended to the user based on the multimodal recommendation model. By employing the above technical solution, frequency domain transformation and phase offset adjustment are performed on the feature vectors of multiple modalities mapped to a unified space to suppress specific noise within a modality, calibrate the phase spectrum offset between modalities, and enhance shared preferences between modalities. This improves the semantic consistency of the recommendation system and enhances the accuracy of item recommendations.

[0055] Please see Figure 2 , Figure 2 This is a flowchart illustrating another embodiment of the item recommendation method proposed in this specification. Specifically: S202: In the item recommendation scenario, determine the original item feature vectors of multiple items in at least two modalities, and map the original item feature vectors to a unified representation space to obtain the initial item feature vectors of the multiple items in each different modality; For details, please refer to the method steps in other embodiments of this specification, which will not be repeated here.

[0056] S204: Convert the initial item feature vectors from the time domain or spatial domain to the frequency domain to obtain the initial frequency domain representations at multiple frequency points under each different mode; For example, transforming the initial item feature vector to the frequency domain essentially decomposes the complex feature signal into a series of sinusoidal components of different frequencies. This transformation can reveal the periodic structure and global dependencies hidden in the original data, making the distribution of features clearer. In the frequency domain, low-frequency components typically correspond to the main outline and core semantics of the signal, while high-frequency components often correspond to details or noise. By obtaining initial frequency domain representations at multiple frequency points, independent and controllable operational dimensions can be provided for subsequent phase adjustment, noise filtering, or feature enhancement, thereby improving the robustness of feature representation.

[0057] S206: In the frequency domain, the initial frequency domain representation of each mode is adjusted by phase distribution offset to obtain the target frequency domain representation, and the target frequency domain representation is converted from the frequency domain to the time domain or the spatial domain to obtain the target item feature vector under each different mode; For example, in the frequency domain, phase determines the structure and shape of a signal, and phase misalignment (i.e., semantic misalignment) often exists between different modes or samples due to differences in acquisition conditions. This misalignment is corrected in the frequency domain by adjusting the phase distribution offset, suppressing random phase fluctuations that represent noise, thereby enhancing the robustness of the features. Subsequently, the processed frequency domain data is converted back to the time or spatial domain to restore it to a conventional feature vector format, so that subsequent recommendation models can directly read and utilize it.

[0058] In one feasible implementation, step S206 includes the following steps: Please see Figure 3 , Figure 3 This is a schematic diagram of a phase distribution offset adjustment process for an item recommendation method proposed in this specification.

[0059] S12: Determine the phase offset of the multiple items at multiple frequency points according to the phase spectrum vector of the multiple items in each different mode, and calibrate the initial frequency domain characterization based on the phase offset to obtain the first intermediate frequency domain characterization. The first intermediate frequency domain representation is a complex frequency domain representation obtained after phase calibration of the initial frequency domain representation. The amplitude spectrum of the initial frequency domain representation remains unchanged, and the phase spectrum has been aligned by the phase offset.

[0060] For example, data acquisition from different modalities often results in phase spectrum deviations (i.e., phase offsets) in the frequency domain. If left unaddressed, these deviations can interfere with the model's understanding of features. Therefore, calculating and calibrating the phase offsets essentially corrects this misalignment, forcibly bringing the features of different modalities to the same baseline. The resulting first intermediate frequency domain representation eliminates systematic errors, ensuring that the information from different modalities of a unified item truly corresponds during subsequent fusion, thereby significantly improving the accuracy of recommendations.

[0061] In one feasible implementation, the at least two modalities include a text modality and an image modality, and step S12 includes the following steps: S122: Obtain the phase spectrum vectors of the multiple items in text mode and image mode; determine the first phase offset of the multiple items at multiple frequency points based on the phase spectrum vector using a first deviation calculation formula; determine the second phase offset based on the first phase offset using a second deviation calculation formula; perform symmetrical phase rotation calibration on the initial frequency domain representation of the text mode and image mode according to the second phase offset; and compensate for the residual deviation of the initial frequency domain representation after symmetrical phase rotation calibration using a calibration calculation formula to obtain the first intermediate frequency domain representation. The first deviation calculation formula satisfies the following formula:

[0062] in, Let i be the first phase offset of the i-th item. Let be the phase spectrum vector of the i-th item in the text modality, and . Let be the phase spectrum vector of the i-th item in the image mode; The second deviation calculation formula satisfies the following formula:

[0063] in, This is the second phase offset; The calibration calculation formula satisfies the following formula:

[0064] in, For the initial frequency domain representation of the i-th item in the image modality, This represents the initial frequency domain representation of the i-th item in the text modality. This is the residual phase compensation vector. This represents the first intermediate frequency domain of the i-th item in the image modality. This represents the first intermediate frequency domain of the i-th item in the text modality.

[0065] S14: Determine the filtering weights of the multiple items at the corresponding multiple frequency points based on the amplitude of the first intermediate frequency domain representation, and adjust the amplitude of the first intermediate frequency domain representation based on the filtering weights to obtain the second intermediate frequency domain representation. The second intermediate frequency domain representation is a complex frequency domain representation obtained by filtering and adjusting the amplitude (amplitude spectrum) of the first intermediate frequency domain representation. The phase spectrum of the first intermediate frequency domain representation remains unchanged, and the amplitude spectrum has been weighted and adjusted by filtering weights.

[0066] For example, features from different modalities often contain noise information irrelevant to the recommendation task. For instance, image modalities may contain visual noise such as background and occlusion, while text modalities may contain semantic noise such as redundant words and irrelevant descriptions. These modality-specific noises typically manifest in the frequency domain as components with large energy fluctuations at certain frequency points but low semantic contributions. If used directly without suppression, this noise will be amplified during subsequent fusion, leading to distorted item representations. Therefore, by analyzing the amplitude statistical distribution of multiple items at the same frequency point, signals (effective semantics) and noise are distinguished. Effective semantic information corresponds to frequency components that are stable across different items, while random noise corresponds to those frequency components that fluctuate irregularly. The filter weights calculated accordingly can automatically identify and retain effective semantic frequency components while suppressing noise-dominated frequency components. The resulting second intermediate frequency domain representation removes modality-specific noise interference, retains effective semantic components that truly reflect item attributes, and provides cleaner feature input for subsequent fusion of different modalities.

[0067] In one feasible implementation, step S14 includes the following steps: S142: Determine the amplitude of the first intermediate frequency domain representation of the multiple items at the same frequency point in the same mode; calculate the median absolute deviation using the median absolute deviation calculation formula based on the amplitude; determine the noise variance using the noise variance calculation formula based on the median absolute deviation; determine the total amplitude variance of the multiple items at the same frequency point; determine the effective semantic variance using the effective semantic variance calculation formula based on the total amplitude variance at the same frequency point and the noise variance; determine the filter weight using the filter weight calculation formula based on the effective semantic variance; adjust the amplitude of the first intermediate frequency domain representation using the amplitude adjustment calculation formula based on the filter weight to obtain the second intermediate frequency domain representation. The formula for calculating the absolute deviation of the median satisfies the following formula:

[0068]

[0069] Where m represents the text modality or image modality, and is... This represents the median absolute deviation of the text or image modality at the f-th frequency point. Let N be the set of amplitudes of all items in the text or image modality at the f-th frequency point, where N is the last item in the set of all items. Let f be the amplitude of the i-th item at the f-th frequency point in either the text or image mode, after symmetric phase rotation calibration. The amplitude of the k-th item at the f-th frequency point in either the text or image mode is calibrated after symmetric phase rotation. The noise variance calculation formula satisfies the following formula:

[0070] in, For text modality or image modality The noise variance at the f-th frequency point This is a constant used to correct for the absolute deviation of the median; The effective semantic variance calculation formula satisfies the following formula:

[0071] in, For text modality or image modality Effective semantic variance at the f-th frequency point For text modality or image modality The total variance of the amplitude at the f-th frequency point; The formula for calculating the filter weights satisfies the following:

[0072] in, For text modality or image modality The filter weights at the f-th frequency point; The amplitude adjustment calculation formula satisfies the following formula:

[0073] in, It represents the second intermediate frequency domain of the i-th item at the f-th frequency point in either the text or image modality.

[0074] S16: Determine the amplitude squared coherence coefficient based on the first intermediate frequency domain representation of the multiple items in each different mode and the amplitude of the first intermediate frequency domain representation, and reweight the first intermediate frequency domain representation based on the amplitude squared coherence coefficient to obtain the third intermediate frequency domain representation. The third intermediate frequency domain representation is a complex frequency domain representation obtained by coherently enhancing the first intermediate frequency domain representation between different modes. The shared frequency components between different modes are weighted and amplified by the amplitude squared coherence coefficient, while the non-shared mode-specific components are suppressed.

[0075] For example, although different modalities reflect the same item, the information they contain is not completely overlapping. In the frequency domain, semantic information reflecting common user preferences often manifests as a collaborative response of different modalities at the same frequency point, while non-shared frequency components contribute little to the recommendation task or even interfere. The amplitude-squared coherence coefficient is an indicator used to quantify the strength of this cross-modal coupling: the larger the coefficient, the stronger the consistency and coupling of the frequency component across different modalities, and the more likely it is to contain shared effective preference information; the smaller the coefficient, the more likely it is to be non-shared noise or irrelevant perturbation. Therefore, by calculating this coefficient and reweighting the first intermediate frequency domain representation, we can effectively suppress modality-specific interference information while retaining and amplifying the consistent components across modalities. The resulting third intermediate frequency domain representation highlights the truly common semantic information between different modalities, providing more discriminative feature inputs for subsequent fusion, thereby improving the recommendation model's ability to capture the consistency of user preferences.

[0076] In one feasible implementation, step S16 includes the following steps: S162: Based on the first intermediate frequency domain characterization of the multiple items in each different mode, the cross-modal cross power spectrum of the items at multiple frequency points is determined by the cross power spectrum calculation formula. Based on the amplitude of the first intermediate frequency domain characterization, the rate spectrum density of the items in each different mode is determined by the power spectrum density calculation formula. Based on the cross-modal cross power spectrum and the rate spectrum density, the amplitude square coherence coefficient of the items at multiple frequency points is determined by the amplitude square coherence coefficient calculation formula. Based on the amplitude square coherence coefficient, the first intermediate frequency domain characterization is reweighted by the reweighting calculation formula to obtain the third intermediate frequency domain characterization. The cross-power spectrum calculation formula satisfies the following formula:

[0077] in, Let be the cross-modal power spectrum of the i-th item at multiple frequency points, denoted by . For element-wise multiplication, For complex conjugate; The power spectral density calculation formula satisfies the following formula:

[0078] in, Let be the power spectral density of the i-th item in the text mode or image mode; The formula for calculating the amplitude squared coherence coefficient satisfies the following formula:

[0079] in, Let be the amplitude squared coherence coefficient of the i-th item at multiple frequency points. For the i-th item Power spectral density in text mode, For the i-th item Power spectral density in image mode; The reweighted calculation formula satisfies the following formula:

[0080] in, It represents the third intermediate frequency domain of the i-th item at multiple frequency points in the text or image modality.

[0081] S18: The second intermediate frequency domain representation and the third intermediate frequency domain representation are weighted and fused to obtain the target frequency domain representation.

[0082] The target frequency domain representation is the final complex frequency domain representation obtained by adaptively weighting and fusing the second intermediate frequency domain representation (after denoising) and the third intermediate frequency domain representation (after coherence enhancement). The contribution ratio of the two types of representations to the final recommendation can be dynamically balanced by learnable parameters.

[0083] For example, the second intermediate frequency domain representation focuses on suppressing modal-specific noise while preserving the effective semantic information of each modality; the third intermediate frequency domain representation focuses on enhancing cross-modal shared preference information, highlighting the consistent response between different modalities. Both have their own emphasis, but neither type of representation can fully cover all the requirements of the recommendation task. For example, overemphasizing denoising may result in the loss of some modality-specific discriminative information, and overemphasizing shared enhancement may ignore modality-specific semantic cues. Therefore, learnable fusion parameters can be introduced, through a soft-maximization function (SoftMaximum). The fusion weights of the two types of representations are normalized, enabling the model to adaptively learn the optimal weight ratio based on specific data features and recommendation tasks. For example, in datasets rich in image information, the denoised second intermediate frequency domain representation can be given higher weights to preserve fine-grained visual features; while in datasets with strong complementarity between image and text information, the coherently enhanced third intermediate frequency domain representation can be given higher weights to highlight cross-modal consistency signals. The resulting target frequency domain representation combines the advantages of noise suppression and shared enhancement, preserving effective intramodal semantics while highlighting cross-modal collaborative responses.

[0084] In one feasible implementation, step S18 includes the following steps: S182: Determine the fusion weights, and use a weighted fusion calculation formula to weight and fuse the second intermediate frequency domain representation and the third intermediate frequency domain representation to obtain the target frequency domain representation. The weighted fusion calculation formula satisfies the following formula:

[0085]

[0086] in, The learnable fusion weights are represented by the second intermediate frequency domain. The learnable fusion weights are represented by the third intermediate frequency domain, and the sum of the fusion weights represented by the second and third intermediate frequency domains is 1. This represents the target frequency domain of the i-th item in either the text or image modal.

[0087] In addition, the feature vector of the target item can be obtained using the following formula.

[0088] in, This represents the inverse fast Fourier transform corresponding to the text modality or image modality. The target item feature vector of the i-th item in either text or image modality.

[0089] S208: Train a multimodal recommendation model based on the feature vector of the target item under each different modality, and perform item recommendation processing on the user based on the multimodal recommendation model.

[0090] For details, please refer to the method steps in other embodiments of this specification, which will not be repeated here.

[0091] In the embodiments of this specification, in an item recommendation scenario, after determining the original item feature vectors of multiple items under various modalities, they are mapped to a unified space to obtain initial item feature vectors. A first frequency domain transformation is performed on the initial item feature vectors to obtain an initial frequency domain representation. A phase distribution offset adjustment is applied to the initial frequency domain representation to obtain a target frequency domain representation, and a second frequency domain transformation is performed to obtain the target item feature vector. A multimodal recommendation model is trained based on the target item feature vector, and items are recommended to the user based on the multimodal recommendation model. By employing the above technical solution, frequency domain transformation and phase offset adjustment are performed on the feature vectors of multiple modalities mapped to a unified space to suppress specific noise within a modality, calibrate the phase spectrum offset between modalities, and enhance shared preferences between modalities. This improves the semantic consistency of the recommendation system and enhances the accuracy of item recommendations.

[0092] Please see Figure 4 , Figure 4 This is a flowchart illustrating another embodiment of the item recommendation method proposed in this specification. Specifically: S301: In an item recommendation scenario, determine the original item feature vectors of multiple items in at least two modalities, and map the original item feature vectors to a unified representation space to obtain the initial item feature vectors of the multiple items in each different modality; For details, please refer to the method steps in other embodiments of this specification, which will not be repeated here.

[0093] S302: Perform a first frequency domain transformation on each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode; For details, please refer to the method steps in other embodiments of this specification, which will not be repeated here.

[0094] S303: In the frequency domain, the initial frequency domain representation of each mode is adjusted by phase distribution offset to obtain the target frequency domain representation, and the target frequency domain representation is transformed by a second frequency domain to obtain the target item feature vector under each different mode; S304: Obtain the user's historical interaction information; For example, the core of an item recommendation system lies in understanding user preferences, and historical interaction information (such as clicks, purchases, and browsing time) is the most direct reflection of a user's true intentions. By acquiring this data, the system can know what users have liked and disliked in the past. In subsequent training, these historical behaviors will serve as supervision signals to guide the model in learning the mapping relationship between item features and user interests.

[0095] S305: Input the feature vector of the target item under each different modality into the multimodal recommendation model; These scattered, calibrated feature vectors of target items are integrated and fed into the core network of the recommendation model. The model needs to receive this information simultaneously to overcome the limitations of a single modality and learn the complementarity and correlation between features from different modalities. Only by inputting these target item feature vectors into the model can it calculate the complex similarities between items and the degree of matching between items and the user's potential interests.

[0096] S306: Use the user's historical interaction information as a supervision signal to train the multimodal recommendation model; Although the model receives the feature vector of the target item, it initially doesn't know which features are important to the user. Using historical interaction information as a supervisory signal (i.e., the training objective), the system compares the model's predictions with the user's actual behavior. Utilizing this difference, the model's internal parameters (weights) are continuously adjusted through backpropagation. This process gradually allows the model to understand how to encourage user interaction.

[0097] S307: Item recommendation processing is performed on users based on the trained multimodal recommendation model.

[0098] For each target user, a matching score is calculated between that user and each item in the candidate item pool. The model comprehensively considers multiple dimensions of information, including visual appeal and textual relevance, to rank the candidate items. Finally, the system displays the highest-scoring item to the user. This step directly determines the user experience and the platform's conversion rate, representing the ultimate realization of the entire technical solution's value.

[0099] The training process of a multimodal recommendation model is explained below: Model creation: Create an initial multimodal recommendation model for item recommendation scenarios based on machine learning models; Sample data acquisition: Acquire a large amount of sample data, including feature vectors of target items under various modalities and users' historical interaction information.

[0100] Sample data labeling: Based on the needs of the item recommendation scenario, a security expert service is introduced to manually label each sample data with corresponding sample tags; the sample tags include positive and negative feedback tags such as whether the user clicked, purchased, or rated the item.

[0101] Model training process: Input sample data into the initial multimodal recommendation model for at least one round of model training to obtain prediction and judgment results; Based on the prediction and judgment results and manually labeled sample labels, use the model loss function to determine the model loss value; Based on the model loss value, adjust the model parameters of the initial multimodal recommendation model until the model training termination condition is met, and obtain the trained multimodal recommendation model.

[0102] Optionally, the model's training termination conditions may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold. Specific training termination conditions can be determined based on actual circumstances and are not specifically limited here.

[0103] It should be noted that the machine learning models involved in one or more embodiments of this specification include, but are not limited to, fitting of one or more of the following machine learning models: Convolutional Neural Network (CNN) model, Deep Neural Network (DNN) model, Recurrent Neural Networks (RNN) model, Embedding model, Gradient Boosting Decision Tree (GBDT) model, Logistic Regression (LR) model, etc.

[0104] In one feasible implementation, step S307 includes the following steps: S22: Input the user item feature vector of the target user and the target item feature vectors of multiple items into the trained multimodal recommendation model; S24: The trained multimodal recommendation model predicts the probability of interaction between the target user and multiple items; S26: Based on the interaction probability, perform item recommendation processing for the target user for the multiple items.

[0105] For example, the feature vector of the target user and multiple feature vectors from the candidate item pool of the multimodal recommendation system are simultaneously input into the model. Upon receiving the user and item vectors, the multimodal recommendation model performs a series of operations to calculate the interaction probability between the user and each candidate item. This interaction probability is typically converted into a probability value (between 0 and 1), representing the likelihood of the user interacting with that item (such as clicking or purchasing). This step transforms the abstract vector into a concrete, comparable numerical value, thus providing a quantitative basis for recommending items to the target user.

[0106] For example, item recommendation based on interaction probabilities is essentially a sorting and filtering decision-making process. The system first sorts the interaction probabilities predicted by the model between the target user and all candidate items in descending order, forming a priority list from high to low. Then, based on business scenario requirements (such as a fixed length of the recommendation list or the display capacity of the user interface), the system selects the Top-N items (e.g., Top-10 or Top-20) from the sorted list. During the filtering process, the system can also introduce post-processing strategies such as deduplication, fragmentation, and diversity control to avoid overly simplistic or repetitive recommendation results. Finally, these filtered and optimized items are assembled into a recommendation result list and returned to the front end via an interface, completing the personalized recommendation for the user.

[0107] In the embodiments of this specification, in an item recommendation scenario, after determining the original item feature vectors of multiple items under various modalities, they are mapped to a unified space to obtain initial item feature vectors. A first frequency domain transformation is performed on the initial item feature vectors to obtain an initial frequency domain representation. A phase distribution offset adjustment is applied to the initial frequency domain representation to obtain a target frequency domain representation, and a second frequency domain transformation is performed to obtain the target item feature vector. A multimodal recommendation model is trained based on the target item feature vector, and items are recommended to the user based on the multimodal recommendation model. By employing the above technical solution, frequency domain transformation and phase offset adjustment are performed on the feature vectors of multiple modalities mapped to a unified space to suppress specific noise within a modality, calibrate the phase spectrum offset between modalities, and enhance shared preferences between modalities. This improves the semantic consistency of the recommendation system and enhances the accuracy of item recommendations.

[0108] The following will combine Figure 5 This specification provides a detailed description of the item recommendation device provided in the embodiments. It should be noted that... Figure 5 The item recommendation device shown is used to perform this instruction manual. Figures 1-4 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 1-4 The example shown.

[0109] Please see Figure 5 This diagram illustrates the structure of an item recommendation device according to an embodiment of this specification. The item recommendation device 1 can be implemented as all or part of a user terminal through software, hardware, or a combination of both. According to some embodiments, the item recommendation device 1 includes a determining module 11, a transforming module 12, an adjusting module 13, and a recommending module 14. The determination module 11 is used to determine the original item feature vectors of multiple items in at least two modalities in an item recommendation scenario, and map the original item feature vectors to a unified representation space to obtain the initial item feature vectors of the multiple items in each different modality. Transformation module 12 is used to perform a first frequency domain transformation on each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode. The adjustment module 13 is used to perform phase distribution offset adjustment on the initial frequency domain representation of each mode in the frequency domain to obtain the target frequency domain representation, and to perform a second frequency domain transformation on the target frequency domain representation to obtain the target item feature vector under each different mode; The recommendation module 14 is used to train a multimodal recommendation model based on the feature vector of the target item under each different modality, and to perform item recommendation processing on the user based on the multimodal recommendation model.

[0110] In one feasible implementation, the first frequency domain transformation of each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode includes: The initial item feature vectors are transformed from the time domain or spatial domain to the frequency domain to obtain the initial frequency domain representations at multiple frequency points under each different mode.

[0111] In one feasible implementation, the step of adjusting the phase distribution offset of the initial frequency domain representation of each mode in the frequency domain to obtain the target frequency domain representation includes: Based on the phase spectrum vector of the multiple items in each different mode, the phase offset of the multiple items at the corresponding multiple frequency points is determined, and the initial frequency domain characterization is calibrated based on the phase offset to obtain the first intermediate frequency domain characterization. The filtering weights of the multiple items at the corresponding multiple frequency points are determined based on the amplitude of the first intermediate frequency domain representation, and the amplitude of the first intermediate frequency domain representation is adjusted based on the filtering weights to obtain the second intermediate frequency domain representation. The amplitude squared coherence coefficient is determined based on the first intermediate frequency domain representation of the multiple items in each different mode and the amplitude of the first intermediate frequency domain representation. The first intermediate frequency domain representation is then reweighted based on the amplitude squared coherence coefficient to obtain the third intermediate frequency domain representation. The target frequency domain representation is obtained by weighted fusion of the second intermediate frequency domain representation and the third intermediate frequency domain representation.

[0112] In one feasible implementation, the at least two modalities include a text modality and an image modality. The step of determining the phase shift of the plurality of items at corresponding frequency points based on the phase spectral vectors of the plurality of items in each different modality, and calibrating the initial frequency domain representation based on the phase shift to obtain a first intermediate frequency domain representation, includes: The phase spectrum vectors of the multiple items in text mode and image mode are obtained. Based on the phase spectrum vectors, the first phase offset of the multiple items at multiple frequency points is determined using a first deviation calculation formula. Based on the first phase offset, the second phase offset is determined using a second deviation calculation formula. According to the second phase offset, the initial frequency domain representations of the text mode and image mode are symmetrically phase rotated and calibrated. The residual deviation of the initial frequency domain representation after symmetrical phase rotation calibration is compensated using a calibration calculation formula to obtain the first intermediate frequency domain representation. The first deviation calculation formula satisfies the following formula:

[0113] in, Let i be the first phase offset of the i-th item. Let be the phase spectrum vector of the i-th item in the text modality, and . Let be the phase spectrum vector of the i-th item in the image mode; The second deviation calculation formula satisfies the following formula:

[0114] in, This is the second phase offset; The calibration calculation formula satisfies the following formula:

[0115] in, For the initial frequency domain representation of the i-th item in the image modality, This represents the initial frequency domain representation of the i-th item in the text modality. This is the residual phase compensation vector. This represents the first intermediate frequency domain of the i-th item in the image modality. This represents the first intermediate frequency domain of the i-th item in the text modality.

[0116] In one feasible implementation, determining the filtering weights of the plurality of items at corresponding frequency points based on the amplitude of the first intermediate frequency domain representation, and adjusting the amplitude of the first intermediate frequency domain representation based on the filtering weights to obtain the second intermediate frequency domain representation, includes: The amplitude of the multiple items represented in the first intermediate frequency domain at the same frequency point in the same mode is determined. The median absolute deviation is calculated using the median absolute deviation calculation formula based on the amplitude. The noise variance is determined using the noise variance calculation formula based on the median absolute deviation. The total amplitude variance of the multiple items at the same frequency point is determined. The effective semantic variance is determined using the effective semantic variance calculation formula based on the effective semantic variance. The filter weight is determined using the filter weight calculation formula based on the filter weight. The amplitude represented in the first intermediate frequency domain is adjusted using the amplitude adjustment calculation formula based on the filter weight to obtain the second intermediate frequency domain representation. The formula for calculating the absolute deviation of the median satisfies the following formula:

[0117]

[0118] Where m represents the text modality or the image modality, This represents the median absolute deviation of the text or image modality at the f-th frequency point. The amplitude of the i-th item at the f-th frequency point in either text or image mode after symmetric phase rotation calibration; The noise variance calculation formula satisfies the following formula:

[0119] in, For text modality or image modality The noise variance at the f-th frequency point This is a constant used to correct for the absolute deviation of the median; The effective semantic variance calculation formula satisfies the following formula:

[0120] in, For text modality or image modality Effective semantic variance at the f-th frequency point For text modality or image modality The total variance of the amplitude at the f-th frequency point; The formula for calculating the filter weights satisfies the following:

[0121] in, For text modality or image modality The filter weights at the f-th frequency point; The amplitude adjustment calculation formula satisfies the following formula:

[0122] in, It represents the second intermediate frequency domain of the i-th item at the f-th frequency point in either the text or image modality.

[0123] In one feasible implementation, the step of determining the amplitude squared coherence coefficient based on the first intermediate frequency domain representation of the plurality of items in each different mode and the amplitude of the first intermediate frequency domain representation, and reweighting the first intermediate frequency domain representation based on the amplitude squared coherence coefficient to obtain the third intermediate frequency domain representation, includes: Based on the first intermediate frequency domain characterization of the multiple items in each different mode, the cross-modal cross power spectrum of the items at multiple frequency points is determined by the cross power spectrum calculation formula. Based on the amplitude of the first intermediate frequency domain characterization, the rate spectral density of the items in each different mode is determined by the power spectral density calculation formula. Based on the cross-modal cross power spectrum and the rate spectral density, the amplitude squared coherence coefficient of the items at multiple frequency points is determined by the amplitude squared coherence coefficient calculation formula. Based on the amplitude squared coherence coefficient, the first intermediate frequency domain characterization is reweighted by the reweighting calculation formula to obtain the third intermediate frequency domain characterization. The cross-power spectrum calculation formula satisfies the following formula:

[0124] in, Let be the cross-modal power spectrum of the i-th item at multiple frequency points, denoted by . For element-wise multiplication, For complex conjugate; The power spectral density calculation formula satisfies the following formula:

[0125] in, Let be the power spectral density of the i-th item in the text mode or image mode; The formula for calculating the amplitude squared coherence coefficient satisfies the following formula:

[0126] in, Let be the amplitude squared coherence coefficient of the i-th item at multiple frequency points. For the i-th item Power spectral density in text mode, For the i-th item Power spectral density in image mode; The reweighted calculation formula satisfies the following formula:

[0127] in, It represents the third intermediate frequency domain of the i-th item at multiple frequency points in the text or image modality.

[0128] In one feasible implementation, the at least two modalities include a text modality and an image modality, and the weighted fusion of the second intermediate frequency domain representation and the third intermediate frequency domain representation to obtain the target frequency domain representation includes: Determine the fusion weights, and use a weighted fusion calculation formula based on the fusion weights to perform weighted fusion of the second intermediate frequency domain representation and the third intermediate frequency domain representation to obtain the target frequency domain representation. The weighted fusion calculation formula satisfies the following formula:

[0129] in, The fusion weights are represented by the second intermediate frequency domain. The fusion weights represented by the third intermediate frequency domain are 1, and the sum of the fusion weights represented by the second and third intermediate frequency domains is 1. This represents the target frequency domain of the i-th item in either the text or image modal.

[0130] In one feasible implementation, the step of performing a second frequency domain transformation on the target frequency domain representation to obtain the target item feature vectors under different modes includes: The target frequency domain representation is transformed from the frequency domain to the time domain or the spatial domain to obtain the target item feature vector under each different mode.

[0131] In one feasible implementation, training a multimodal recommendation model based on the target item feature vectors under different modalities, and performing item recommendation processing on the user based on the multimodal recommendation model, includes: Obtain the user's historical interaction information; The feature vectors of the target items under each different modality are input into the multimodal recommendation model; The user's historical interaction information is used as a supervision signal to train the multimodal recommendation model; The trained multimodal recommendation model is used to recommend items to users.

[0132] In one feasible implementation, the method of recommending items to a user based on the trained multimodal recommendation model includes: The user item feature vector of the target user and the target item feature vectors of multiple items are input into the trained multimodal recommendation model; The trained multimodal recommendation model predicts the probability of interaction between the target user and multiple items. Based on the interaction probability, item recommendation processing is performed on the target user for the multiple items.

[0133] It should be noted that the item recommendation device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the item recommendation method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the item recommendation device and the item recommendation method embodiments provided in the above embodiments belong to the same concept, and the implementation process can be found in the method embodiments, which will not be repeated here.

[0134] The example numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the examples.

[0135] In the embodiments of this specification, in an item recommendation scenario, after determining the original item feature vectors of multiple items under various modalities, they are mapped to a unified space to obtain initial item feature vectors. A first frequency domain transformation is performed on the initial item feature vectors to obtain an initial frequency domain representation. A phase distribution offset adjustment is applied to the initial frequency domain representation to obtain a target frequency domain representation, and a second frequency domain transformation is performed to obtain the target item feature vector. A multimodal recommendation model is trained based on the target item feature vector, and items are recommended to the user based on the multimodal recommendation model. By employing the above technical solution, frequency domain transformation and phase offset adjustment are performed on the feature vectors of multiple modalities mapped to a unified space to suppress specific noise within a modality, calibrate the phase spectrum offset between modalities, and enhance shared preferences between modalities. This improves the semantic consistency of the recommendation system and enhances the accuracy of item recommendations.

[0136] This specification also provides a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1-4 The item recommendation method described in the illustrated embodiment can be found in the following documentation for its specific execution process. Figures 1-4 The specific details of the illustrated embodiments will not be elaborated here.

[0137] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figures 1-4 The item recommendation method described in the illustrated embodiment can be found in the following documentation for its specific execution process. Figures 1-4 The specific details of the illustrated embodiments will not be elaborated here.

[0138] Please refer to Figure 6 This diagram illustrates a structural block diagram of an electronic device provided in an exemplary embodiment of this specification. The electronic device in this specification may include one or more components such as a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected via the bus 150.

[0139] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 110 may integrate one or more of the following: central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0140] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described below, etc. The operating system may be the Android system, including systems deeply developed based on the Android system, the iOS system developed by Apple Inc., including systems deeply developed based on the iOS system, or other systems. The data storage area may also store data created by the electronic device during use, such as phonebook data, audio and video data, chat log data, etc.

[0141] See Figure 7As shown, the memory 120 can be divided into operating system space and user space. The operating system runs in the operating system space, while native and third-party applications run in the user space. To ensure that different third-party applications can achieve good running performance, the operating system allocates corresponding system resources for each application. However, different application scenarios within the same third-party application have different requirements for system resources. For example, in local resource loading scenarios, third-party applications have high requirements for disk read speed; in animation rendering scenarios, third-party applications have high requirements for GPU performance. Since the operating system and third-party applications are independent of each other, the operating system often cannot promptly perceive the current application scenario of a third-party application, resulting in the operating system's inability to adapt system resources accordingly to the specific application scenario of the third-party application.

[0142] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0143] Taking the Android operating system as an example, the programs and data stored in memory 120 are as follows: Figure 8As shown, the memory 120 can store the Linux kernel layer 320, the system runtime library layer 340, the application framework layer 360, and the application layer 380. The Linux kernel layer 320, system runtime library layer 340, and application framework layer 360 belong to the operating system space, while the application layer 380 belongs to the user space. The Linux kernel layer 320 provides low-level drivers for various hardware components of the electronic device, such as display drivers, audio drivers, camera drivers, Bluetooth drivers, Wi-Fi drivers, and power management. The system runtime library layer 340 provides support for key features of the Android system through several C / C++ libraries. For example, the SQLite library provides database support, the OpenGL / ES library provides 3D graphics support, and the Webkit library provides browser kernel support. The system runtime library layer 340 also provides the Android runtime library, which mainly provides core libraries that allow developers to write Android applications using the Java language. The Application Framework Layer 360 provides various APIs that may be used when building applications. Developers can also use these APIs to build their own applications, such as activity management, window management, view management, notification management, content provider, package management, call management, resource management, and location management. At least one application runs in the Application Layer 380. These applications can be native applications that come with the operating system, such as contacts, SMS, clock, and camera apps; or third-party applications developed by third-party developers, such as games, instant messaging, and photo editing apps.

[0144] Taking the operating system as an example (iOS), the programs and data stored in memory 120 are as follows: Figure 9As shown, the iOS system includes: Core OS layer 420, Core Services layer 440, Media layer 460, and Cocoa Touch layer 480. Core OS layer 420 includes the operating system kernel, drivers, and low-level program frameworks. These low-level program frameworks provide hardware-level functionality for use by the program frameworks located in Core Services layer 440. Core Services layer 440 provides system services and / or program frameworks required by applications, such as Foundation framework, account framework, advertising framework, data storage framework, network connectivity framework, geolocation framework, motion framework, etc. Media layer 460 provides applications with audiovisual interfaces, such as interfaces related to graphics and images, audio technology, video technology, and wireless playback (AirPlay) interfaces. Cocoa Touch layer 480 provides various commonly used interface-related frameworks for application development and is responsible for user touch interaction on electronic devices. Examples include local notification services, remote push services, advertising frameworks, game tool frameworks, message user interface (UI) frameworks, UIKit user interface frameworks, map frameworks, and so on.

[0145] exist Figure 9 The framework shown includes, but is not limited to, the base framework in the core service layer 440 and the UIKit framework in the touchable layer 480. The base framework provides many basic object classes and data types, offering the most basic system services to all applications, and is independent of the UI. The UIKit framework, on the other hand, provides a basic UI class library for creating touch-based user interfaces. iOS applications can use the UIKit framework to provide their UI, thus providing the application's infrastructure for building user interfaces, drawing, handling user interaction events, responding to gestures, and so on.

[0146] The methods and principles for implementing data communication between third-party applications and the operating system in the iOS system can be found in the Android system, and will not be repeated here.

[0147] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined into a touch screen, which is used to receive touch operations from the user using a finger, stylus, or any suitable object on or near it, and to display the user interface of various applications. The touch screen is usually located on the front panel of the electronic device. The touch screen can be designed as a full-screen, curved screen, or irregularly shaped screen. The touch screen can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen; this specification does not limit this aspect.

[0148] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WiFi) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0149] In the embodiments of this specification, the executing entity for each step can be the electronic device described above. Optionally, the executing entity for each step can be the operating system of the electronic device. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0150] The electronic device described in this specification can also be equipped with a display device. This display device can be any device capable of displaying information, such as a cathode ray tube display (CR), a light-emitting diode display (LED), an e-ink screen, a liquid crystal display (LCD), or a plasma display panel (PDP). Users can use the display device on the electronic device to view displayed text, images, videos, and other information. The electronic device can be a smartphone, tablet computer, gaming device, AR (Augmented Reality) device, automobile, data storage device, audio playback device, video playback device, laptop, desktop computing device, or wearable device such as a smartwatch, smart glasses, smart helmet, smart bracelet, smart necklace, or smart clothing.

[0151] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0152] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. A method for recommending items, characterized in that, The method includes: In the item recommendation scenario, the original item feature vectors of multiple items in at least two modalities are determined, and the original item feature vectors are mapped to a unified representation space to obtain the initial item feature vectors of the multiple items in each different modality. Perform a first frequency domain transformation on each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode; In the frequency domain, the initial frequency domain representation of each mode is phase-distribution offset adjusted to obtain the target frequency domain representation, and the target frequency domain representation is subjected to a second frequency domain transformation to obtain the target item feature vector under each different mode; A multimodal recommendation model is trained based on the feature vector of the target item under each different modality, and the user is recommended items based on the multimodal recommendation model.

2. The method according to claim 1, characterized in that, The step of performing a first frequency domain transformation on each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode includes: The initial item feature vectors are transformed from the time domain or spatial domain to the frequency domain to obtain the initial frequency domain representations at multiple frequency points under each different mode.

3. The method according to claim 1, characterized in that, The step of adjusting the phase distribution offset of the initial frequency domain representation of each mode in the frequency domain to obtain the target frequency domain representation includes: Based on the phase spectrum vector of the multiple items in each different mode, the phase offset of the multiple items at the corresponding multiple frequency points is determined, and the initial frequency domain characterization is calibrated based on the phase offset to obtain the first intermediate frequency domain characterization. The filtering weights of the multiple items at the corresponding multiple frequency points are determined based on the amplitude of the first intermediate frequency domain representation, and the amplitude of the first intermediate frequency domain representation is adjusted based on the filtering weights to obtain the second intermediate frequency domain representation. The amplitude squared coherence coefficient is determined based on the first intermediate frequency domain representation of the multiple items in each different mode and the amplitude of the first intermediate frequency domain representation. The first intermediate frequency domain representation is then reweighted based on the amplitude squared coherence coefficient to obtain the third intermediate frequency domain representation. The target frequency domain representation is obtained by weighted fusion of the second intermediate frequency domain representation and the third intermediate frequency domain representation.

4. The method according to claim 3, characterized in that, The at least two modalities include a text modality and an image modality. The step of determining the phase shift of the multiple items at corresponding frequency points based on the phase spectral vectors of the multiple items in each different modality, and calibrating the initial frequency domain representation based on the phase shift to obtain a first intermediate frequency domain representation, includes: The phase spectrum vectors of the multiple items in text mode and image mode are obtained. Based on the phase spectrum vectors, the first phase offset of the multiple items at multiple frequency points is determined using a first deviation calculation formula. Based on the first phase offset, the second phase offset is determined using a second deviation calculation formula. According to the second phase offset, the initial frequency domain representations of the text mode and image mode are symmetrically phase rotated and calibrated. The residual deviation of the initial frequency domain representation after symmetrical phase rotation calibration is compensated using a calibration calculation formula to obtain the first intermediate frequency domain representation. The first deviation calculation formula satisfies the following formula: in, Let i be the first phase offset of the i-th item. Let be the phase spectrum vector of the i-th item in the text modality, and . Let be the phase spectrum vector of the i-th item in the image mode; The second deviation calculation formula satisfies the following formula: in, This is the second phase offset; The calibration calculation formula satisfies the following formula: in, For the initial frequency domain representation of the i-th item in the image modality, This represents the initial frequency domain representation of the i-th item in the text modality. This is the residual phase compensation vector. This represents the first intermediate frequency domain of the i-th item in the image modality. This represents the first intermediate frequency domain of the i-th item in the text modality.

5. The method according to claim 4, characterized in that, The step of determining the filtering weights of the multiple items at corresponding multiple frequency points based on the amplitude of the first intermediate frequency domain representation, and adjusting the amplitude of the first intermediate frequency domain representation based on the filtering weights to obtain the second intermediate frequency domain representation includes: The amplitude of the multiple items represented in the first intermediate frequency domain at the same frequency point in the same mode is determined. The median absolute deviation is calculated using the median absolute deviation calculation formula based on the amplitude. The noise variance is determined using the noise variance calculation formula based on the median absolute deviation. The total amplitude variance of the multiple items at the same frequency point is determined. The effective semantic variance is determined using the effective semantic variance calculation formula based on the effective semantic variance. The filter weight is determined using the filter weight calculation formula based on the filter weight. The amplitude represented in the first intermediate frequency domain is adjusted using the amplitude adjustment calculation formula based on the filter weight to obtain the second intermediate frequency domain representation. The formula for calculating the absolute deviation of the median satisfies the following formula: Where m represents the text modality or the image modality, This represents the median absolute deviation of the text or image modality at the f-th frequency point. Let f be the set of amplitude values ​​of all items in the text or image modality at the f-th frequency point. Let f be the amplitude of the i-th item at the f-th frequency point in either the text or image mode, after symmetric phase rotation calibration. The amplitude of the k-th item at the f-th frequency point in either the text or image mode is calibrated after symmetric phase rotation. The noise variance calculation formula satisfies the following formula: in, For text modality or image modality The noise variance at the f-th frequency point This is a constant used to correct for the absolute deviation of the median; The effective semantic variance calculation formula satisfies the following formula: in, For text modality or image modality Effective semantic variance at the f-th frequency point For text modality or image modality The total variance of the amplitude at the f-th frequency point; The formula for calculating the filter weights satisfies the following: in, For text modality or image modality The filter weights at the f-th frequency point; The amplitude adjustment calculation formula satisfies the following formula: in, It represents the second intermediate frequency domain of the i-th item at the f-th frequency point in either the text or image modality.

6. The method according to claim 4, characterized in that, The step of determining the amplitude squared coherence coefficient based on the first intermediate frequency domain representation of the multiple items in each different mode, and the amplitude of the first intermediate frequency domain representation, and then reweighting the first intermediate frequency domain representation based on the amplitude squared coherence coefficient to obtain the third intermediate frequency domain representation, includes: Based on the first intermediate frequency domain characterization of the multiple items in each different mode, the cross-modal cross power spectrum of the items at multiple frequency points is determined by the cross power spectrum calculation formula. Based on the amplitude of the first intermediate frequency domain characterization, the rate spectral density of the items in each different mode is determined by the power spectral density calculation formula. Based on the cross-modal cross power spectrum and the rate spectral density, the amplitude squared coherence coefficient of the items at multiple frequency points is determined by the amplitude squared coherence coefficient calculation formula. Based on the amplitude squared coherence coefficient, the first intermediate frequency domain characterization is reweighted by the reweighting calculation formula to obtain the third intermediate frequency domain characterization. The cross-power spectrum calculation formula satisfies the following formula: in, Let be the cross-modal power spectrum of the i-th item at multiple frequency points, denoted by . For element-wise multiplication, For complex conjugate; The power spectral density calculation formula satisfies the following formula: in, Let be the power spectral density of the i-th item in the text mode or image mode; The formula for calculating the amplitude squared coherence coefficient satisfies the following formula: in, Let be the amplitude squared coherence coefficient of the i-th item at multiple frequency points. For the i-th item Power spectral density in text mode, For the i-th item Power spectral density in image mode; The reweighted calculation formula satisfies the following formula: in, It represents the third intermediate frequency domain of the i-th item at multiple frequency points in the text or image modality.

7. The method according to claim 3, characterized in that, The at least two modalities include a text modality and an image modality. The step of weightedly fusing the second intermediate frequency domain representation and the third intermediate frequency domain representation to obtain the target frequency domain representation includes: Determine the fusion weights, and use a weighted fusion calculation formula based on the fusion weights to perform weighted fusion of the second intermediate frequency domain representation and the third intermediate frequency domain representation to obtain the target frequency domain representation. The weighted fusion calculation formula satisfies the following formula: in, The fusion weights are represented by the second intermediate frequency domain. The fusion weights represented by the third intermediate frequency domain are 1, and the sum of the fusion weights represented by the second and third intermediate frequency domains is 1. This represents the target frequency domain of the i-th item in either the text or image modal.

8. The method according to claim 1, characterized in that, The step of performing a second frequency domain transformation on the target frequency domain representation to obtain the target item feature vectors under different modes includes: The target frequency domain representation is transformed from the frequency domain to the time domain or the spatial domain to obtain the target item feature vector under each different mode.

9. The method according to claim 1, characterized in that, The step of training a multimodal recommendation model based on the feature vectors of target items under different modalities, and then performing item recommendation processing on the user based on the multimodal recommendation model, includes: Obtain the user's historical interaction information; The feature vectors of the target items under each different modality are input into the multimodal recommendation model; The user's historical interaction information is used as a supervision signal to train the multimodal recommendation model; The trained multimodal recommendation model is used to recommend items to users.

10. The method according to claim 9, characterized in that, The multimodal recommendation model, based on training, recommends items to the user, including: The user item feature vector of the target user and the target item feature vectors of multiple items are input into the trained multimodal recommendation model; The trained multimodal recommendation model predicts the probability of interaction between the target user and multiple items. Based on the interaction probability, item recommendation processing is performed on the target user for the multiple items.

11. An item recommendation device, characterized in that, The device includes: The determination module is used to determine the original item feature vectors of multiple items in at least two modalities in an item recommendation scenario, and to map the original item feature vectors to a unified representation space to obtain the initial item feature vectors of the multiple items in each different modality. The transformation module is used to perform a first frequency domain transformation on each of the initial item feature vectors to obtain the initial frequency domain representation at multiple frequency points under each different mode. The adjustment module is used to perform phase distribution offset adjustment on the initial frequency domain representation of each mode in the frequency domain to obtain the target frequency domain representation, and to perform a second frequency domain transformation on the target frequency domain representation to obtain the target item feature vector under each different mode; The recommendation module is used to train a multimodal recommendation model based on the feature vector of the target item under each different modality, and to perform item recommendation processing for users based on the multimodal recommendation model.

12. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions, which are adapted to be loaded by a processor and executed as method steps as claimed in any one of claims 1 to 10.

13. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed the method steps as claimed in any one of claims 1 to 10.

Images