Commodity recommendation method, device and equipment and readable storage medium

Abstract

The application discloses a commodity recommendation method, device and equipment and a readable storage medium. The method comprises the following steps: determining an emotional label of a user based on multi-modal data of the user; determining a product recommendation strategy corresponding to the emotional label and a shopping stage currently experienced by the user based on the emotional label and the shopping stage; determining virtual commodities contained in a current shopping interface based on the product recommendation strategy; determining a rendering effect of the current shopping interface and / or a rendering effect of the virtual commodities in the current shopping interface based on the emotional label; and displaying the current shopping interface according to the determined rendering effect. The method can improve the accuracy of emotion recognition, determine a product recommendation strategy according to the determined emotional label and the shopping stage currently experienced by the user, make the recommendation timing more accurate, match the rendering effect with the emotional label, enhance the interactive feedback, and significantly improve the immersion, decision efficiency and satisfaction of the user in the virtual shopping process.

Description

Technical Field

[0001] This application belongs to the field of communication technology, and specifically relates to a product recommendation method, apparatus, device, and readable storage medium. Background Technology

[0002] Current virtual product recommendation methods rely solely on a single visual modality (facial image), which is susceptible to lighting, occlusion, and the user's deliberate control of facial expressions; or they rely solely on a single EEG modality, which is costly, inconvenient to wear, and the signal is easily interfered with by noise. This results in biased judgment of emotional state, susceptibility to interference, and limited accuracy, leading to a discrepancy between the recommended virtual products and the user's immediate needs and emotional inclinations, and insufficient personalization. Summary of the Invention

[0003] This application provides a product recommendation method, apparatus, device, and readable storage medium to address the problem that current technology-recommended virtual products deviate from users' immediate needs and emotional preferences, and lack sufficient personalization.

[0004] Firstly, a product recommendation method is provided, the method comprising:

[0005] Based on the user's multimodal data, determine the user's emotion label; the emotion label includes at least one of the following: excitement; interest; hesitation; anxiety; satisfaction.

[0006] Based on the emotion tag and the user's current shopping stage, a product recommendation strategy corresponding to the emotion tag and the shopping stage is determined; wherein, the shopping stage includes at least one of the following: browsing stage, decision-making stage, and payment stage;

[0007] Based on the product recommendation strategy, determine the virtual products included in the current shopping interface;

[0008] Based on the emotion tags, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

[0009] The current shopping interface will be displayed according to the determined rendering effect.

[0010] The step of determining the rendering effect of the current shopping interface and / or the rendering effect of virtual goods on the current shopping interface based on the emotion tags includes:

[0011] Based on the emotion tag, determine the rendering parameters corresponding to the emotion tag;

[0012] Based on the rendering parameters, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

[0013] The determination of a user's emotion tag based on the user's multimodal data includes:

[0014] The multimodal data is input into a multi-head self-attention mechanism to generate a fused emotion vector;

[0015] The user's emotion tag is determined based on the quantized value of the fused emotion vector and the trigger threshold matched with the shopping stage; wherein the trigger threshold is different for different shopping stages.

[0016] The method further includes:

[0017] Collect multimodal data of users, including at least one of the following: electroencephalogram (EEG) data, eye movement data, facial expression data, and speech data.

[0018] The method further includes:

[0019] Collect data from the entire shopping process;

[0020] The end-to-end data is input into a machine learning model to obtain the degree of influence of different emotion tags output by the machine learning model on conversion rate, satisfaction rate and repurchase rate.

[0021] Based on the degree of impact, dynamically adjust at least one of the following: rendering parameters, product recommendation strategy, and trigger threshold matching the shopping stage.

[0022] Secondly, embodiments of this application also provide a product recommendation method based on group sentiment, the method further comprising:

[0023] Based on multimodal data from multiple users, emotional tags for the same target virtual product are determined; the emotional tags include at least one of the following: excitement; interest; hesitation; anxiety; satisfaction.

[0024] If the sentiment tags of multiple users for the same target virtual product meet the preset group sentiment consistency condition, a group recommendation identifier is added to the target virtual product, and / or the ranking weight of the target virtual product in multiple user interfaces is increased.

[0025] The preset group emotional consistency conditions include:

[0026] Multiple users were labeled as excited or interested, and their gaze duration exceeded the first preset duration.

[0027] The method further includes:

[0028] The multimodal data of each user is input into a multi-head self-attention mechanism to generate a fused sentiment vector for each user.

[0029] Based on the quantified value of each user's fused emotion vector and the trigger threshold matched with the shopping stage, the emotion label of each user is determined; the trigger threshold is different for different shopping stages.

[0030] The method further includes:

[0031] Collect multimodal data from multiple users, including at least one of the following: electroencephalogram (EEG) data, eye-tracking data, facial expression data, and speech data.

[0032] Thirdly, embodiments of this application also provide a product recommendation device, including a processor and a transceiver, wherein the transceiver receives and transmits data under the control of the processor, and the processor is used to perform the following operations:

[0033] Based on the user's multimodal data, determine the user's emotion label; the emotion label includes at least one of the following: excitement; interest; hesitation; anxiety; satisfaction.

[0034] Based on the emotion tag and the user's current shopping stage, a product recommendation strategy corresponding to the emotion tag and the shopping stage is determined; wherein, the shopping stage includes at least one of the following: browsing stage, decision-making stage, and payment stage;

[0035] Based on the product recommendation strategy, determine the virtual products included in the current shopping interface;

[0036] Based on the emotion tags, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

[0037] The current shopping interface will be displayed according to the determined rendering effect.

[0038] The processor is also used to perform the following operations:

[0039] Based on the emotion tag, determine the rendering parameters corresponding to the emotion tag;

[0040] Based on the rendering parameters, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

[0041] The processor is also used to perform the following operations:

[0042] The multimodal data is input into a multi-head self-attention mechanism to generate a fused emotion vector;

[0043] The user's emotion tag is determined based on the quantized value of the fused emotion vector and the trigger threshold matched with the shopping stage; wherein the trigger threshold is different for different shopping stages.

[0044] The processor is also used to perform the following operations:

[0045] Collect multimodal data of users, including at least one of the following: electroencephalogram (EEG) data, eye movement data, facial expression data, and speech data.

[0046] The processor is also used to perform the following operations:

[0047] Collect data from the entire shopping process;

[0048] The end-to-end data is input into a machine learning model to obtain the degree of influence of different emotion tags output by the machine learning model on conversion rate, satisfaction rate and repurchase rate.

[0049] Based on the degree of impact, dynamically adjust at least one of the following: rendering parameters, product recommendation strategy, and trigger threshold matching the shopping stage.

[0050] Fourthly, embodiments of this application also provide a product recommendation device based on group sentiment, including a processor and a transceiver. The transceiver receives and sends data under the control of the processor, and the processor is used to perform the following operations:

[0051] Based on multimodal data from multiple users, emotional tags for the same target virtual product are determined; the emotional tags include at least one of the following: excitement; interest; hesitation; anxiety; satisfaction.

[0052] If the sentiment tags of multiple users for the same target virtual product meet the preset group sentiment consistency condition, a group recommendation identifier is added to the target virtual product, and / or the ranking weight of the target virtual product in multiple user interfaces is increased.

[0053] The preset group emotional consistency conditions include:

[0054] Multiple users were labeled as excited or interested, and their gaze duration exceeded the first preset duration.

[0055] This application also provides a product recommendation device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the product recommendation method described above.

[0056] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the product recommendation method described above.

[0057] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the product recommendation method described above.

[0058] The above-mentioned technical solution of this application has at least the following beneficial effects:

[0059] In the product recommendation method, apparatus, device, and readable storage medium of this application embodiment, the user's emotion tags are perceived through the user's multimodal data, which can improve the accuracy of emotion recognition; and the product recommendation strategy is determined based on the determined emotion tags and the user's current shopping stage, making the recommendation timing more accurate; and the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface are determined based on the product recommendation strategy and emotion tags; the rendering effect matches the emotion tags, which can enhance interactive feedback, thereby significantly improving the user's immersion, decision-making efficiency, and satisfaction in the virtual shopping process. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating the steps of the product recommendation method provided in the embodiments of this application;

[0061] Figure 2 One of the example diagrams illustrates the product recommendation method provided in the embodiments of this application;

[0062] Figure 3 The second example diagram illustrates the product recommendation method provided in this application embodiment;

[0063] Figure 4 This is a flowchart illustrating the steps of the product recommendation method based on group sentiment provided in an embodiment of this application.

[0064] Figure 5 The third example diagram illustrates the product recommendation method provided in this application embodiment;

[0065] Figure 6 Figure 4 illustrates the product recommendation method provided in this application embodiment;

[0066] Figure 7 This is a schematic diagram of one of the product recommendation devices provided in the embodiments of this application;

[0067] Figure 8 This is a second schematic diagram illustrating the structure of the product recommendation device provided in the embodiments of this application. Detailed Implementation

[0068] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0069] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and are not used to describe a specified order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, a first object can be one or more. Furthermore, in the specification and claims, "and" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0070] It is worth noting that the technologies described in this application are not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA), and other systems. The terms "system" and "network" in this application are often used interchangeably, and the described technologies can be used with the systems and radio technologies mentioned above, as well as with other systems and radio technologies. However, the following description describes New Radio (NR) systems for illustrative purposes, and NR terminology is used in most of the following description. These technologies can also be applied to applications beyond NR systems, such as 6th Generation (6G) communication systems.

[0071] like Figure 1 As shown in the figure, this application provides a product recommendation method, the method including:

[0072] Step 101: Based on the user's multimodal data, determine the user's emotion label; the emotion label includes at least one of the following: excitement; interest; hesitation; anxiety; satisfaction;

[0073] Alternatively, a user's emotion label can also be referred to as the user's emotional state.

[0074] Alternatively, the excited state can also be called the positive state, the interested state can also be called the focused state, and the anxious state can also be called the negative state, etc., without making specific limitations here.

[0075] Step 102: Based on the user's emotion tag and current shopping stage, determine a product recommendation strategy corresponding to the emotion tag and the shopping stage; wherein the shopping stage includes at least one of the following: browsing stage, decision-making stage, and payment stage;

[0076] Step 103: Determine the virtual products included in the current shopping interface according to the product recommendation strategy.

[0077] Step 104: Based on the emotion tags, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

[0078] Optionally, a user's emotion tags can be dynamically updated, such as in real time, periodically, or triggered by tag changes or specific events; no specific limitations are imposed here.

[0079] Step 105: Display the current shopping interface according to the determined rendering effect.

[0080] This application's embodiments target Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) environments, and bind the rendering effects of virtual goods or shopping interfaces to the user's emotion tags in real time, thereby achieving dynamic adjustment of the rendering effects of virtual goods or shopping interfaces.

[0081] In one implementation, step 102 includes:

[0082] Based on the emotion tag, determine the rendering parameters corresponding to the emotion tag;

[0083] Based on the rendering parameters, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

[0084] Optionally, rendering parameters include, but are not limited to, color saturation, brightness, contrast, dynamic particle effect type and intensity, and information tag content and style.

[0085] Example 1: The user's emotion label is excited / positive. The corresponding rendering effects include: enhancing the lighting effects of virtual goods (brightness +30%, contrast +20%), adding dynamic particle effects (such as starlight and halo), and playing upbeat sound effects.

[0086] Example 2: The user's emotion label is focused / interested. The corresponding rendering effects include: smoothly zooming in on the product area that the user is looking at, and providing a floating layer with more detailed information.

[0087] Example 3: The user's emotional label is hesitant. The corresponding rendering effects include: highlighting key attributes (magnifying material texture by 2 times), overlaying comparative information labels (such as cost-effectiveness and functional differences compared to similar products), and adopting a stable and clear rendering style.

[0088] Example 4: The user's emotion label is negative / anxious. The corresponding rendering effects include: a softer interface color scheme, reduced visual distractions, and clear and concise operation guidance.

[0089] This application embodiment binds and dynamically adjusts the user's real-time emotion tags / states with the rendering parameters of virtual goods / environments in real time; optionally, specific rendering effects are displayed for different emotion tags (excitement, focus, hesitation, negativity). This emotional rendering enables the virtual environment to dynamically respond to the user's emotions, providing highly personalized visual and interactive feedback, greatly enhancing the sense of presence and participation.

[0090] This application embodiment accurately perceives the user's emotion tags through the user's multimodal data, and determines the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface based on the determined emotion tags; the rendering effect matches the emotion tags, thereby significantly improving the user's immersion, decision-making efficiency and satisfaction in the virtual shopping process.

[0091] In one implementation, when a user's emotion label is excitement or interest and the intensity exceeds a certain threshold, the product recommendation strategy includes:

[0092] 1) Recommend high-premium products (new products, limited editions, designer items).

[0093] 2) Recommended product price range increased (e.g., +20%-30%).

[0094] 3) Products that meet the above conditions will be displayed first in the recommendation list.

[0095] In another implementation, when the user's emotional label is hesitant or anxious, the product recommendation strategy includes:

[0096] 1) Recommend soothing / practical / high-value products; such as aromatherapy, snacks, basic items, discounted items - reference solution - negative response.

[0097] 2) Recommend the price range of the products to be reduced (e.g., -15% to -20%).

[0098] 3) Proactively push coupons, promotional information, or products with a historical positive review rate of >90% (to enhance trust).

[0099] For example, using eye-tracking data from multimodal data, for products where the user's gaze duration is significantly higher than average or the "intensity of interest" is high, the recommendation strategy is to increase the ranking weight in the recommendation list or to label them (such as labeling them "you pay more attention to them").

[0100] For example, during the browsing stage, the recommendation strategy focuses on diverse exploration recommendations to stimulate interest. During the decision / comparison stage, the recommendation strategy includes: proactively recommending comparable products (feature and price comparison views), or highlighting key decision-making information (such as zoomed-in details + comparison labels: "Ranked #1 in value for money," etc.). The post-payment recommendation strategy involves recommending complementary products, peripheral items, or maintenance services.

[0101] This application's embodiments are based on quantifying emotional intensity (such as EEG beta wave energy, eye movement interest intensity, and facial expression probability) and duration, and dynamically setting and migrating trigger thresholds in conjunction with the current shopping stage (browsing / decision-making / payment). Based on these dynamic trigger thresholds, a differentiated recommendation strategy is implemented (e.g., recommending high-priced new products in a positive state, recommending promotional / soothing products in a negative state, and providing comparative information in a decision-making state). Specific emotion tags drive dynamic adjustments to product price ranges (e.g., percentage increase / decrease). Thus, the end-to-end latency from simultaneous multimodal signal acquisition, fusion analysis to driving recommendation and rendering updates is <150ms, far superior to traditional recommendation systems that rely on historical data or offline analysis (typically at the second level), representing an improvement of over 50%. The recommended content is highly matched to the user's immediate emotional needs.

[0102] In this embodiment, a product recommendation strategy is comprehensively determined based on the user's different emotional tags and current shopping stage, thereby recommending virtual products that match the user's emotional tags and current shopping stage. This method significantly enhances the user's immersive shopping experience, personalized experience, and decision-making efficiency by deeply understanding the user's real-time emotional state and intentions in the virtual shopping environment and dynamically adjusting the product recommendation strategy and virtual rendering effects according to the shopping stage.

[0103] In at least one embodiment of this application, determining the user's emotion tag based on the user's multimodal data includes:

[0104] The multimodal data is input into a multi-head self-attention mechanism to generate a fused emotion vector;

[0105] The user's emotion tag is determined based on the quantized value of the fused emotion vector and the trigger threshold matched with the shopping stage; wherein the trigger threshold is different for different shopping stages.

[0106] In this process, each modal data point (such as EEG feature vectors, eye-tracking feature vectors, facial expression feature vectors, and speech feature vectors) is mapped to a unified modal feature vector (e.g., a 256-dimensional modal feature vector) through an independent encoder sub-network. The modal feature vector is then input into a multi-head self-attention mechanism to calculate the interrelationships and importance weights between the modal features. The attention weights are used to fuse the information from each modality, generating a fused emotion vector (e.g., a 512-dimensional fused emotion vector). This fused emotion vector is then processed by a classification function (e.g., Softmax) to output real-time, fine-grained user emotion labels, such as "highly positive - excited," "moderately positive - interested," "neutral - browsing," "mildly negative - hesitant," and "highly negative - anxious."

[0107] To achieve different trigger thresholds for different shopping stages, this application introduces a dynamic emotion state machine. First, key emotion tags related to the shopping experience are defined, such as excitement, focus / interest, hesitation, anxiety, and satisfaction. Then, the classification probability value or specific feature value (such as EEG excitation index or eye-tracking interest intensity) of the fused emotion vector is quantified to obtain the quantified value of the fused emotion vector; and the duration of specific emotion tags is tracked. Dynamic trigger thresholds are set according to different shopping stages.

[0108] This application utilizes the Transformer architecture to perform real-time synchronous, cross-modal correlation analysis and fusion of EEG signal features, eye-tracking features (including fixation point and pupil changes), facial expression features (including macro-expressions and micro-expressions), and speech emotion features to generate unified fine-grained emotion labels. Through multimodal (EEG + eye-tracking + facial expression + speech) complementary fusion, it overcomes the limitations of a single signal source (such as facial camouflage and EEG noise), achieving an emotion classification accuracy of ≥92%.

[0109] In one implementation, the excited state is defined as: (EEG beta wave energy ≥ X μV² / Hz) AND (eye movement interest intensity ≥ Y) AND (probability of "happy" expression ≥ Z%). The decision-making stage threshold can be reduced by 20% to more sensitively capture purchase signals.

[0110] In another implementation, negative / anxious states are assessed using either verbal "anxiety" probability ≥ A% AND facial "negative" probability ≥ B% or EEG-specific anxiety patterns. The threshold can be increased during the payment phase to avoid misjudgments.

[0111] For example, such as Figure 2 As shown, the threshold rules for triggering the excited state during the browsing phase are: X≥0.8μV² / Hz, Y≥3.0; correspondingly, the triggering threshold for the excited state during the decision-making phase is automatically reduced by 20%; the composite conditions for triggering the anxious state during the decision-making phase are: voice anxiety≥70%, negative facial expression≥60%; or a specific EEG anxiety pattern.

[0112] It should be noted that the embodiments of this application also provide a state transition method, that is, to drive the transition of system state (and corresponding recommendation strategy) based on the real-time fused emotion tags, intensity, duration and current shopping stage.

[0113] In at least one embodiment of this application, the method further includes:

[0114] Collect multimodal data of users, including at least one of the following: electroencephalogram (EEG) data, eye movement data, facial expression data, and speech data.

[0115] For example, a multimodal emotion perception layer is used to collect multi-source physiological and behavioral signals (i.e., multimodal data) from users in a real-time, synchronous manner in a VR / AR / MR environment. This multimodal emotion perception layer includes at least one of the following modules: an electroencephalogram (EEG) signal acquisition module, an eye-tracking module, a facial expression recognition module, and a facial expression identification module.

[0116] EEG signal acquisition module:

[0117] Portable flexible EEG caps (e.g., 32-channel) cover key brain regions (Fp1, Fp2, C3, C4, etc.), supporting high sampling rates (≥1000Hz) and high signal-to-noise ratios (>80dB). An integrated bioelectrical signal acquisition chip is used. FPGA performs real-time bandpass filtering (0.5-40Hz), downsampling, and noise reduction (refer to ICA / filtering in Scheme 2). Key feature extraction: Calculate the power spectral density (PSD) of specific frequency bands (δ, θ, α, β, γ) to generate feature vectors (e.g., excitation index = β-wave energy / α-wave energy, focus level = θ-wave energy).

[0118] Eye-tracking module:

[0119] Based on a near-infrared camera (120fps+) and a high-precision PCCR algorithm, the offset between the pupil center and the corneal reflection point is calculated to achieve gaze tracking (accuracy error <0.5°). The "interest intensity" index is defined as gaze duration (seconds) × pupil diameter change rate (%). A threshold (e.g., ≥2.5) is set to trigger high-priority recommendations or identify highly interesting products.

[0120] Facial expression recognition module:

[0121] Facial images are captured using a high-resolution camera. A lightweight CNN model (such as MobileNetV3) is deployed, with input consisting of coordinates of 68 facial key points and an RGB image. The model is trained on an expanded dataset (such as FER-2013 + micro-expression labels) and outputs a fine-grained emotion probability distribution (e.g., happy: 0.82, hesitant: 0.15, disgust: 0.03 - the category of expanded scheme one). Inference latency is <50ms, supporting operation on mobile / VR devices.

[0122] Voice sentiment analysis module:

[0123] Real-time analysis of user voice interaction content extracts MFCC coefficients, fundamental frequency (F0) and its standard deviation, speech rate, pause frequency, etc., to construct a 128-dimensional emotion feature vector. Based on the LSTM model, real-time emotion classification (such as pleasure, anxiety, neutrality, dissatisfaction, excitement) is performed on the speech, with an accuracy of ≥89%.

[0124] It should be noted that all sensor data are strictly synchronized using a unified timestamp to ensure that multimodal data are aligned in time.

[0125] In at least one embodiment of this application, the method further includes:

[0126] Collect data from the entire shopping process;

[0127] The end-to-end data is input into a machine learning model to obtain the degree of influence of different emotion tags output by the machine learning model on conversion rate, satisfaction rate and repurchase rate.

[0128] Based on the degree of impact, dynamically adjust at least one of the following: rendering parameters, product recommendation strategy, and trigger threshold matching the shopping stage.

[0129] For example, such as Figure 4 As shown, this application embodiment tracks end-to-end user data from emotion-triggered recommendation -> interaction behavior (click, add to cart) -> purchase decision -> post-receipt feedback (rating, review, repurchase). A post-purchase emotion decay model is constructed (e.g., assessing the daily change rate of user excitement / satisfaction through subsequent interactions or questionnaires after receipt). High satisfaction standards are defined (e.g., decay rate <5% / day). Machine learning models (e.g., logistic regression, gradient boosting trees) are used to analyze the impact of different emotional states and recommendation strategies on the final conversion rate, satisfaction, and repurchase rate, dynamically optimizing strategy parameters in the recommendation engine (e.g., trigger thresholds for various emotion tags, price fluctuation ratios, product type weights, etc.).

[0130] In at least one embodiment of this application, such as Figure 3As shown, in the online environment, different recommendation strategy versions (such as different combinations of trigger thresholds for emotion tags, rendering effects, etc.) run in parallel, such as strategy version A and strategy version B. Key metrics (such as click-through rate, add-to-cart rate, purchase conversion rate, and dwell time) are monitored in real time. Furthermore, the decision engine automatically selects the strategy with the highest emotion matching degree and the best conversion effect based on data performance and deploys it to the production environment for promotion.

[0131] In summary, the product recommendation method provided in this application provides an emotion-adaptive recommendation system that accurately matches immediate needs, expected to reduce user decision-making time by 30%; the immersive emotional experience and "me-aware" recommendations significantly enhance user experience, with an expected increase in satisfaction of 40%; recommendations based on real-time interests (eye movement) and emotions help users discover desired products they might have overlooked; high-matching recommendations significantly improve click-through rates, conversion rates, and average order value, with promotional activity conversion rates expected to increase by 50%; more accurate demand forecasting and recommendations help reduce unsold inventory by 20%; the emotion-driven, high-premium new product recommendation mechanism accelerates new product market penetration; and the unique immersive experience attracts and retains users.

[0132] like Figure 4 As shown in the embodiments of this application, a product recommendation method based on group sentiment is also provided, the method further comprising:

[0133] Step 401: Based on multimodal data from multiple users, determine the emotional tags of multiple users for the same target virtual product; the emotional tags include at least one of the following: excitement; interest; hesitation; anxiety; satisfaction.

[0134] Step 402: If the sentiment tags of multiple users for the same target virtual product meet the preset group sentiment consistency condition, add a group recommendation identifier to the target virtual product and / or increase the ranking weight of the target virtual product in multiple user interfaces.

[0135] In one implementation, the preset group emotional consistency condition includes: multiple users' emotional labels are excited or interested, and the gaze duration is greater than a first preset duration.

[0136] For example, such as Figure 5 As shown, in a multi-person VR shopping scenario, the system analyzes the emotional reactions of a group towards the same product or area in real time. If it detects that ≥N people (e.g., 3 people) have a "positive" emotional label for the same product and their gaze duration is >T seconds (e.g., 5 seconds), the system automatically adds a "group recommendation" label to the product and significantly increases its ranking weight (e.g., +200%) in the interfaces of all participants, and may trigger a social sharing prompt.

[0137] In at least one embodiment of this application, the method further includes:

[0138] The multimodal data of each user is input into a multi-head self-attention mechanism to generate a fused sentiment vector for each user.

[0139] Based on the quantified value of each user's fused emotion vector and the trigger threshold matched with the shopping stage, the emotion label of each user is determined; the trigger threshold is different for different shopping stages.

[0140] In this process, each modal data point (such as EEG feature vectors, eye-tracking feature vectors, facial expression feature vectors, and speech feature vectors) is mapped to a unified modal feature vector (e.g., a 256-dimensional modal feature vector) through an independent encoder sub-network. The modal feature vector is then input into a multi-head self-attention mechanism to calculate the interrelationships and importance weights between the modal features. The attention weights are used to fuse the information from each modality, generating a fused emotion vector (e.g., a 512-dimensional fused emotion vector). This fused emotion vector is then processed by a classification function (e.g., Softmax) to output real-time, fine-grained user emotion labels, such as "highly positive - excited," "moderately positive - interested," "neutral - browsing," "mildly negative - hesitant," and "highly negative - anxious."

[0141] To achieve different trigger thresholds for different shopping stages, this application introduces a dynamic emotion state machine. First, key emotion tags related to the shopping experience are defined, such as excitement, focus / interest, hesitation, anxiety, and satisfaction. Then, the classification probability value or specific feature value (such as EEG excitation index or eye-tracking interest intensity) of the fused emotion vector is quantified to obtain the quantified value of the fused emotion vector; and the duration of specific emotion tags is tracked. Dynamic trigger thresholds are set according to different shopping stages.

[0142] This application utilizes the Transformer architecture to perform real-time synchronous, cross-modal correlation analysis and fusion of EEG signal features, eye-tracking features (including fixation point and pupil changes), facial expression features (including macro-expressions and micro-expressions), and speech emotion features to generate unified fine-grained emotion labels. Through multimodal (EEG + eye-tracking + facial expression + speech) complementary fusion, it overcomes the limitations of a single signal source (such as facial camouflage and EEG noise), achieving an emotion classification accuracy of ≥92%.

[0143] In one implementation, the excited state is defined as: (EEG beta wave energy ≥ X μV² / Hz) AND (eye movement interest intensity ≥ Y) AND (probability of "happy" expression ≥ Z%). The decision-making stage threshold can be reduced by 20% to more sensitively capture purchase signals.

[0144] In another implementation, negative / anxious states are assessed using either verbal "anxiety" probability ≥ A% AND facial "negative" probability ≥ B% or EEG-specific anxiety patterns. The threshold can be increased during the payment phase to avoid misjudgments.

[0145] In at least one embodiment of this application, the method further includes:

[0146] Collect multimodal data from multiple users, including at least one of the following: electroencephalogram (EEG) data, eye-tracking data, facial expression data, and speech data.

[0147] For example, a multimodal emotion perception layer is used to collect multi-source physiological and behavioral signals (i.e., multimodal data) from users in a real-time, synchronous manner in a VR / AR / MR environment. This multimodal emotion perception layer includes at least one of the following modules: an electroencephalogram (EEG) signal acquisition module, an eye-tracking module, a facial expression recognition module, and a facial expression identification module.

[0148] EEG signal acquisition module:

[0149] Portable flexible EEG caps (e.g., 32-channel) cover key brain regions (Fp1, Fp2, C3, C4, etc.), supporting high sampling rates (≥1000Hz) and high signal-to-noise ratios (>80dB). An integrated bioelectrical signal acquisition chip is used. FPGA performs real-time bandpass filtering (0.5-40Hz), downsampling, and noise reduction (refer to ICA / filtering in Scheme 2). Key feature extraction: Calculate the power spectral density (PSD) of specific frequency bands (δ, θ, α, β, γ) to generate feature vectors (e.g., excitation index = β-wave energy / α-wave energy, focus level = θ-wave energy).

[0150] Eye-tracking module:

[0151] Based on a near-infrared camera (120fps+) and a high-precision PCCR algorithm, the offset between the pupil center and the corneal reflection point is calculated to achieve gaze tracking (accuracy error <0.5°). The "interest intensity" index is defined as gaze duration (seconds) × pupil diameter change rate (%). A threshold (e.g., ≥2.5) is set to trigger high-priority recommendations or identify highly interesting products.

[0152] Facial expression recognition module:

[0153] Facial images are captured using a high-resolution camera. A lightweight CNN model (such as MobileNetV3) is deployed, with input consisting of coordinates of 68 facial key points and an RGB image. The model is trained on an expanded dataset (such as FER-2013 + micro-expression labels) and outputs a fine-grained emotion probability distribution (e.g., happy: 0.82, hesitant: 0.15, disgust: 0.03 - the category of expanded scheme one). Inference latency is <50ms, supporting operation on mobile / VR devices.

[0154] Voice sentiment analysis module:

[0155] Real-time analysis of user voice interaction content extracts MFCC coefficients, fundamental frequency (F0) and its standard deviation, speech rate, pause frequency, etc., to construct a 128-dimensional emotion feature vector. Based on the LSTM model, real-time emotion classification (such as pleasure, anxiety, neutrality, dissatisfaction, excitement) is performed on the speech, with an accuracy of ≥89%.

[0156] It should be noted that all sensor data are strictly synchronized using a unified timestamp to ensure that multimodal data are aligned in time.

[0157] In summary, in multi-user VR / AR / MR collaborative shopping scenarios, this application embodiment aggregates and analyzes the emotional state and eye-tracking attention data of multiple users towards the same target product / area in real time. Based on preset group emotional consistency conditions (such as ≥N people in a positive state and gaze > T seconds), it triggers a "group recommendation" flag, improves social decision-making efficiency, and significantly enhances product ranking weight and social prompts.

[0158] To better describe the product recommendation method provided in the embodiments of this application, an example is given below, such as... Figure 6 As shown, the product recommendation method includes:

[0159] VR devices acquire multimodal data from users, including: the time spent looking at product A, smiling expressions, the voice saying "This is good", and changes in brain signals.

[0160] The VR device preprocesses the multimodal data and sends it to the fusion engine; specifically, it sends eye-tracking data (e.g., interest intensity = 3.2), facial expression data (e.g., happiness probability = 0.85), voice data (e.g., pleasure classification), and EEG data (e.g., β / α = 1.2).

[0161] The fusion engine performs Transformer fusion processing on the received data to generate emotion labels; for example, the emotion label is highly positive-excited.

[0162] When the state machine detects that the current stage is the decision-making stage, it applies the trigger threshold corresponding to the decision-making stage, and triggers a positive state strategy based on the sentiment tag generated by the fusion engine and the recommendation engine.

[0163] The recommendation engine executes a premium recommendation strategy; for example, it sends instruction 1 to the rendering engine, indicating a recommended price range of +30%; or it sends instruction 2 to the rendering engine, indicating that limited edition items should be displayed first.

[0164] The rendering engine applies lighting and / or adds particle animations based on instructions from the recommendation engine, and updates the product display to VR devices;

[0165] The VR device displays an enhanced version of product A; when the user adds product A to their shopping cart, the VR device further records the conversion behavior of product A.

[0166] In summary, the product recommendation method provided in this application, based on emotion-driven personalized recommendations, increases the click-through rate of target products by over 35% and the purchase conversion rate by over 25%. It accurately matches users' willingness to pay (dynamically adjusted price range) and scenario needs (stage-based strategies), effectively improving average order value and customer satisfaction. Furthermore, it intelligently adapts to changing needs at different shopping stages (browsing / decision / payment) and effectively supports multi-person social shopping scenarios, improving recommendation matching accuracy and scenario applicability by over 40%. Moreover, through emotion-behavior correlation analysis and A / B testing, it ensures that the method's performance continuously improves over time, forming a competitive advantage.

[0167] like Figure 7 As shown in the illustration, this application also provides a product recommendation device, including a processor 700 and a transceiver 710. The transceiver 710 receives and transmits data under the control of the processor 700, and the processor 700 is used to perform the following operations:

[0168] Based on the user's multimodal data, determine the user's emotion label; the emotion label includes at least one of the following: excitement; interest; hesitation; anxiety; satisfaction.

[0169] Based on the user's emotion tag and current shopping stage, a product recommendation strategy corresponding to the emotion tag and shopping stage is determined; wherein, the shopping stage includes at least one of the following: browsing stage, decision-making stage, and payment stage;

[0170] Based on the product recommendation strategy, determine the virtual products included in the current shopping interface;

[0171] Based on the emotion tags, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

[0172] The current shopping interface will be displayed according to the determined rendering effect.

[0173] In some embodiments of this application, the processor is also configured to perform the following operations:

[0174] Based on the emotion tag, determine the rendering parameters corresponding to the emotion tag;

[0175] Based on the rendering parameters, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

[0176] In some embodiments of this application, the processor is also configured to perform the following operations:

[0177] The multimodal data is input into a multi-head self-attention mechanism to generate a fused emotion vector;

[0178] The user's emotion tag is determined based on the quantized value of the fused emotion vector and the trigger threshold matched with the shopping stage; wherein the trigger threshold is different for different shopping stages.

[0179] In some embodiments of this application, the processor is also configured to perform the following operations:

[0180] Collect multimodal data of users, including at least one of the following: electroencephalogram (EEG) data, eye movement data, facial expression data, and speech data.

[0181] In some embodiments of this application, the processor is also configured to perform the following operations:

[0182] Collect data from the entire shopping process;

[0183] The end-to-end data is input into a machine learning model to obtain the degree of influence of different emotion tags output by the machine learning model on conversion rate, satisfaction rate and repurchase rate.

[0184] Based on the degree of impact, dynamically adjust at least one of the following: rendering parameters, product recommendation strategy, and trigger threshold matching the shopping stage.

[0185] This application embodiment constructs an intelligent shopping system that is deeply integrated into the VR / AR / MR environment, can accurately perceive the user's multi-dimensional emotional state in real time, and dynamically drive personalized product recommendation strategies and immersive interactive experiences accordingly, so as to significantly improve the user's immersion, decision-making efficiency and satisfaction in the virtual shopping process.

[0186] like Figure 8 As shown in the illustration, this application also provides a product recommendation device based on group sentiment, including a processor 800 and a transceiver 810. The transceiver 810 receives and sends data under the control of the processor 800, and the processor 800 is used to perform the following operations:

[0187] Based on multimodal data from multiple users, emotional tags for the same target virtual product are determined; the emotional tags include at least one of the following: excitement; interest; hesitation; anxiety; satisfaction.

[0188] If the sentiment tags of multiple users for the same target virtual product meet the preset group sentiment consistency condition, a group recommendation identifier is added to the target virtual product, and / or the ranking weight of the target virtual product in multiple user interfaces is increased.

[0189] In at least one embodiment of this application, the preset group emotional consistency condition includes:

[0190] Multiple users were labeled as excited or interested, and their gaze duration exceeded the first preset duration.

[0191] In at least one embodiment of this application, the processor is further configured to perform the following operations:

[0192] The multimodal data of each user is input into a multi-head self-attention mechanism to generate a fused sentiment vector for each user.

[0193] Based on the quantified value of each user's fused emotion vector and the trigger threshold matched with the shopping stage, the emotion label of each user is determined; the trigger threshold is different for different shopping stages.

[0194] In at least one embodiment of this application, the processor is further configured to perform the following operations:

[0195] Collect multimodal data from multiple users, including at least one of the following: electroencephalogram (EEG) data, eye-tracking data, facial expression data, and speech data.

[0196] In a multi-user VR / AR / MR collaborative shopping scenario, this application embodiment aggregates and analyzes the emotional state and eye-tracking attention data of multiple users toward the same target product / area in real time. Based on preset group emotional consistency conditions (such as ≥N people in a positive state and gaze > T seconds), it triggers a "group recommendation" flag, improves social decision-making efficiency, and significantly enhances product ranking weight and social prompts.

[0197] It should be noted that the product recommendation device provided in this application is a device capable of executing the above-described product recommendation method. Therefore, all embodiments of the above-described product recommendation method are applicable to this device and can achieve the same or similar beneficial effects. No specific limitations are made here.

[0198] This application also provides a product recommendation device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the various processes in the product recommendation method embodiments described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0199] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the various processes in the product recommendation method embodiments described above and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0200] This application also provides a computer program product, including computer instructions. When executed by a processor, these computer instructions implement the various processes of the product recommendation method embodiment described above and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0201] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0202] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 A device for one or more processes and / or the functions specified in one or more boxes.

[0203] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce a paper article including an instruction means, the instruction means being implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment, causing the computer or other programmable equipment to perform a series of operational steps to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0205] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A product recommendation method, characterized in that, The method includes: Based on the user's multimodal data, determine the user's emotion label; the emotion label includes at least one of the following: excitement; interest; hesitation; anxiety; satisfaction. Based on the emotion tag and the user's current shopping stage, a product recommendation strategy corresponding to the emotion tag and the shopping stage is determined; wherein, the shopping stage includes at least one of the following: browsing stage, decision-making stage, and payment stage; Based on the product recommendation strategy, determine the virtual products included in the current shopping interface; Based on the emotion tags, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface. The current shopping interface will be displayed according to the determined rendering effect.

2. The method according to claim 1, characterized in that, The step of determining the rendering effect of the current shopping interface and / or the rendering effect of the virtual goods on the current shopping interface based on the emotion tags includes: Based on the emotion tag, determine the rendering parameters corresponding to the emotion tag; Based on the rendering parameters, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

3. The method according to claim 1, characterized in that, The process of determining a user's emotion tag based on their multimodal data includes: The multimodal data is input into a multi-head self-attention mechanism to generate a fused emotion vector; The user's emotion tag is determined based on the quantized value of the fused emotion vector and the trigger threshold matched with the shopping stage; wherein the trigger threshold is different for different shopping stages.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Collect multimodal data of users, including at least one of the following: electroencephalogram (EEG) data, eye movement data, facial expression data, and speech data.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Collect data from the entire shopping process; The end-to-end data is input into a machine learning model to obtain the degree of influence of different emotion tags output by the machine learning model on conversion rate, satisfaction rate and repurchase rate. Based on the degree of impact, dynamically adjust at least one of the following: rendering parameters, product recommendation strategy, and trigger threshold matching the shopping stage.

6. A product recommendation method based on group sentiment, characterized in that, The method further includes: Based on multimodal data from multiple users, emotional tags for the same target virtual product are determined; the emotional tags include at least one of the following: excitement; interest; hesitation; anxiety; satisfaction. If the sentiment tags of multiple users for the same target virtual product meet the preset group sentiment consistency condition, a group recommendation identifier is added to the target virtual product, and / or the ranking weight of the target virtual product in multiple user interfaces is increased.

7. The method according to claim 6, characterized in that, The preset group emotional consistency conditions include: Multiple users were labeled as excited or interested, and their gaze duration exceeded the first preset duration.

8. The method according to claim 6 or 7, characterized in that, The method further includes: The multimodal data of each user is input into a multi-head self-attention mechanism to generate a fused sentiment vector for each user. Based on the quantified value of each user's fused emotion vector and the trigger threshold matched with the shopping stage, the emotion label of each user is determined; the trigger threshold is different for different shopping stages.

9. The method according to claim 6, characterized in that, The method further includes: Collect multimodal data from multiple users, including at least one of the following: electroencephalogram (EEG) data, eye-tracking data, facial expression data, and speech data.

10. A product recommendation device, comprising a processor and a transceiver, wherein the transceiver receives and transmits data under the control of the processor, characterized in that, The processor is used to perform the following operations: Based on the user's multimodal data, determine the user's emotion label; the emotion label includes at least one of the following: excitement; interest; hesitation; anxiety; satisfaction. Based on the emotion tag and the user's current shopping stage, a product recommendation strategy corresponding to the emotion tag and the shopping stage is determined; wherein, the shopping stage includes at least one of the following: browsing stage, decision-making stage, and payment stage; Based on the product recommendation strategy, determine the virtual products included in the current shopping interface; Based on the emotion tags, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface. The current shopping interface will be displayed according to the determined rendering effect.

11. The product recommendation device according to claim 10, characterized in that, The processor is also used to perform the following operations: Based on the emotion tag, determine the rendering parameters corresponding to the emotion tag; Based on the rendering parameters, determine the rendering effect of the current shopping interface and / or the rendering effect of the virtual products on the current shopping interface.

12. The product recommendation device according to claim 10, characterized in that, The processor is also used to perform the following operations: The multimodal data is input into a multi-head self-attention mechanism to generate a fused emotion vector; The user's emotion tag is determined based on the quantized value of the fused emotion vector and the trigger threshold matched with the shopping stage; wherein the trigger threshold is different for different shopping stages.

13. The product recommendation device according to any one of claims 10-12, characterized in that, The processor is also used to perform the following operations: Collect multimodal data of users, including at least one of the following: electroencephalogram (EEG) data, eye movement data, facial expression data, and speech data.

14. The product recommendation device according to any one of claims 10-13, characterized in that, The processor is also used to perform the following operations: Collect data from the entire shopping process; The end-to-end data is input into a machine learning model to obtain the degree of influence of different emotion tags output by the machine learning model on conversion rate, satisfaction rate and repurchase rate. Based on the degree of impact, dynamically adjust at least one of the following: rendering parameters, product recommendation strategy, and trigger threshold matching the shopping stage.

15. A product recommendation device based on group sentiment, comprising a processor and a transceiver, wherein the transceiver receives and transmits data under the control of the processor, characterized in that, The processor is used to perform the following operations: Based on multimodal data from multiple users, emotional tags for the same target virtual product are determined; the emotional tags include at least one of the following: excitement; interest; hesitation; anxiety; satisfaction. If the sentiment tags of multiple users for the same target virtual product meet the preset group sentiment consistency condition, a group recommendation identifier is added to the target virtual product, and / or the ranking weight of the target virtual product in multiple user interfaces is increased.

16. The apparatus according to claim 15, characterized in that, The preset group emotional consistency conditions include: Multiple users were labeled as excited or interested, and their gaze duration exceeded the first preset duration.

17. A product recommendation device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that, When the processor executes the program, it implements the product recommendation method as described in any one of claims 1-5, or the product recommendation method based on group sentiment as described in any one of claims 6-9.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the product recommendation method as described in any one of claims 1-5, or the steps of the product recommendation method based on group sentiment as described in any one of claims 6-9.

19. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the steps of the product recommendation method as described in any one of claims 1-5, or implement the steps of the product recommendation method based on group sentiment as described in any one of claims 6-9.

Images