Immersive merchandise information interaction display system and method

Abstract

This invention discloses an immersive product information interactive display system and method, relating to the field of virtual reality technology. It trains a neural collaborative filtering model by using user behavior feature vectors, cluster feature labels, and product comprehensive feature vectors as inputs, and the predicted values ​​of product interest labels as outputs. The model collects the behavior feature vectors of users to be interacted with, as well as the comprehensive feature vectors of currently displayed products. Based on the behavior feature vectors and user behavior clusters, it obtains the target cluster label for the user to be interacted with. Based on the behavior feature vectors, target cluster labels, comprehensive feature vectors, and the neural collaborative filtering model, it generates a detailed display decision. This achieves the goal of analyzing the correlation between behavioral patterns and product features, gaining a deeper understanding of influencing factors, and thus more proactively controlling and intervening in recommendation strategies.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, specifically to an immersive product information interactive display system and method. Background Technology

[0002] With the development of technology, traditional product display methods can no longer meet consumer needs. Virtual reality (VR) and augmented reality (AR) technologies can provide immersive user experiences, offering new possibilities for product display. However, current VR and AR product displays focus primarily on visual effects, neglecting the interactivity between users and products. For example, due to the large number of products, users may browse products they are interested in but find it difficult to actively seek further information, thus missing out on products they are interested in. Therefore, a technological solution is needed that can determine whether a user is interested in a product and proactively display product details based on that determination.

[0003] Chinese Patent CN114708044B, authorized by the Ministry of Industry and Information Technology, discloses a method, apparatus, and electronic device for processing virtual item information and training a model. The method involves acquiring textual descriptions, image information, metadata, and interaction logs associated with virtual items; extracting key elements from the image information to obtain key image features of the virtual items; extracting attribute features, market features, seller features, and other feature information based on the interaction logs and metadata; extracting multimodal features of the virtual items; and using a trained deep learning network to accurately determine the value information of the virtual items based on these multimodal features. However, this method fails to determine the user's interest state in a virtual reality scene.

[0004] Therefore, this invention proposes an immersive product information interactive display system and method. Summary of the Invention

[0005] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention proposes an immersive product information interactive display system and method. By analyzing the correlation between behavioral patterns and product characteristics, it achieves a deeper understanding of influencing factors, thereby enabling more proactive control and intervention in recommendation strategies.

[0006] To achieve the above objectives, an immersive product information interactive display method is proposed, including the following steps:

[0007] Step 1: Collect historical user behavior data from N test users in advance from the virtual device, convert the historical user behavior data into user behavior feature vectors, collect visual data and text data of various products, and collect product interest tags for each test user; N is the preset number of test users;

[0008] Step 2: Cluster the user behavior feature vectors using a clustering algorithm to divide the test users into K user behavior clusters, where K is the preset number of user behavior clusters; the cluster number to which each test user belongs is used as the cluster feature label for the test user.

[0009] Step 3: Use a computer vision model to extract visual feature vectors from the product visual data; use a text analysis model to extract semantic feature vectors from the product text data; and combine the visual and semantic features of each product category to form a comprehensive product feature vector.

[0010] Step 4: Use the user's user behavior feature vector, cluster feature label, and product comprehensive feature vector as input to the neural collaborative filtering model, and use the predicted value of the product interest label as output to train the neural collaborative filtering model.

[0011] Step 5: Collect the behavioral feature vectors of the users to be interacted with, as well as the comprehensive feature vectors of the currently displayed products;

[0012] Step 6: Based on behavioral feature vectors and user behavior clusters, obtain the target cluster label of the user to be interacted with; based on behavioral feature vectors, target cluster labels, comprehensive feature vectors, and a neural collaborative filtering model, generate a details display decision;

[0013] The process of pre-collecting historical user behavior data from N test users from virtual devices includes the following steps:

[0014] Step 11: Recruit N volunteers as test users and build a product display space in the virtual reality engine;

[0015] Step 12: Display 3D rendered models of various products for each test user in the product display space, and record the raw behavioral data of each test user in real time through the device's sensors and tracking system;

[0016] Step 13: Use each tester's raw behavioral data as user historical behavioral data;

[0017] The method for collecting visual and textual data of various commodities is as follows:

[0018] Collect images of various products displayed in the product display space as product visual data;

[0019] Collect text descriptions of various products displayed alongside product images in the product display space as product text data;

[0020] The product interest tags are entered by each test user after each display of a type of product. The product interest tag is 0 or 1. When the product interest tag is 0, it means that the test user is not interested in the type of product. When the product interest tag is 1, it means that the test user is interested in the type of product.

[0021] The step of using a clustering algorithm to cluster user behavior feature vectors and dividing test users into K user behavior clusters includes the following steps:

[0022] Step 21: Treat each user behavior feature vector as an M-dimensional discrete point; where M is the dimension of the user behavior feature vector, and the coordinates of the M-dimensional discrete point in each dimension correspond to a user behavior feature in the user behavior feature vector.

[0023] Step 22: Randomly select K M-dimensional discrete points as initial cluster centers, and the remaining M-dimensional discrete points as non-initial cluster centers;

[0024] Step 23: For each non-initial cluster center, calculate the Euclidean distance to each initial cluster center, and assign each M-dimensional discrete point to the cluster containing the nearest initial cluster center;

[0025] Step 24: Calculate the user behavior features corresponding to each dimension of the M-dimensional discrete points within each cluster, and form a new M-dimensional discrete point by averaging the user behavior features corresponding to each dimension. Use this new M-dimensional discrete point as the new cluster center of the cluster.

[0026] Step 25: Recalculate the Euclidean distance from each M-dimensional discrete point to each new cluster center, and reassign each M-dimensional discrete point to the cluster containing the nearest new cluster center;

[0027] Step 26: Repeat steps 24-25 until the M-dimensional discrete points in all the divided clusters no longer change. Label each cluster with the number k, k = 1, 2, 3, ..., K. Obtain all the M-dimensional discrete points contained in each cluster and form a discrete point set from all the M-dimensional discrete points. The k-th user behavior cluster includes the M-dimensional spatial coordinates of the new cluster center of the k-th cluster and the user behavior feature vector corresponding to each M-dimensional discrete point in the k-th cluster.

[0028] The method for extracting product visual feature vectors from product visual data using a computer vision model is as follows:

[0029] Use a pre-trained graph processing neural network model to extract the visual feature vector of the product visual data;

[0030] After obtaining the product visual feature vector, principal component analysis is used to project the high-dimensional product visual features to a low dimension, thereby obtaining a product visual feature vector with a lower dimension.

[0031] The method for extracting semantic feature vectors of products using a text analysis model from product text data is as follows:

[0032] The product text data is segmented into word sequences according to word boundaries using a word segmentation tool, such as jieba.

[0033] The bag-of-words model is used to represent each word in the word sequence as a word frequency vector. The weight of each word is calculated based on the inverse document frequency. The Word2Vec neural network is used to map each word to a fixed-dimensional vector space according to its weight. The BERT model is used to transform the set of vector spaces corresponding to the word sequence into text feature vectors based on the context words. These text feature vectors are the product semantic feature vectors.

[0034] The method for training the neural collaborative filtering model is as follows:

[0035] The neural collaborative filtering model is constructed as follows:

[0036] The neural collaborative filtering model includes an input layer, an embedding layer, an encoding layer, a fully connected layer, and an output layer.

[0037] The input layer includes a first input layer and a second input layer;

[0038] The first input layer is used to receive the user feature vector of the test user, which is composed of user behavior feature vector and clustering feature label;

[0039] The second input layer is used to receive the comprehensive feature vector of the product;

[0040] The embedding layer includes a first embedding layer and a second embedding layer. The first embedding layer is used to map a one-dimensional user feature vector to a low-dimensional dense vector, and the second embedding layer is used to map a product comprehensive feature vector to a corresponding low-dimensional dense vector.

[0041] The encoding layer includes a first encoding layer and a second encoding layer. The first encoding layer encodes the user embedding vector output by the first embedding layer using a multilayer perceptron or a transformer. The second encoding layer encodes the product embedding vector output by the second embedding layer using a multilayer perceptron or a transformer.

[0042] The fully connected layer is used to perform further non-linear encoding on the concatenated vector of the user encoding vector output by the first encoding layer and the product encoding vector output by the second encoding layer;

[0043] The output layer is used to output the predicted values ​​of product interest tags;

[0044] The loss function of the neural collaborative filtering model is -∑ L [y×log(p)+(1-y)×log(1-p)], where y is the product interest tag corresponding to each input group, p is the predicted value of the product interest tag corresponding to each input group, and L is the total amount of data input into the neural collaborative filtering model;

[0045] The user feature vector and the comprehensive product feature vector of each test user are input into the neural collaborative filtering model, and the neural collaborative filtering model is trained until the loss function converges.

[0046] The method for generating detail display decisions based on behavioral feature vectors, target cluster labels, comprehensive feature vectors, and a neural collaborative filtering model is as follows:

[0047] The behavioral feature vector and the target cluster label are combined to form the target user feature vector. The target user feature vector and the comprehensive feature vector are then input into the neural collaborative filtering model to obtain the predicted value of the product interest label output by the neural collaborative filtering model.

[0048] Retrieve the display strategy corresponding to the predicted value of the product interest tag from the preset display strategy set, and execute the display strategy; the display strategy is the result of the details display decision.

[0049] An immersive product information interactive display system is proposed, comprising a training data collection module, a feature vector extraction module, a model training module, and an interactive display module; wherein the modules are connected electrically.

[0050] The training data collection module collects historical user behavior data from N test users in advance from the virtual device, transforms the historical user behavior data into user behavior feature vectors, collects visual data and text data of various products, collects product interest tags for each test user, and sends the user behavior feature vectors, visual data and text data of products to the feature vector extraction module, and sends the product interest tags to the model training module.

[0051] The feature vector extraction module uses a clustering algorithm to cluster user behavior feature vectors, dividing test users into K user behavior clusters. The cluster number of each test user serves as the cluster feature label for the test user. It uses a computer vision model to extract visual feature vectors from product visual data and a text analysis model to extract semantic feature vectors from product text data. The module then combines the visual and semantic features of each product category to form a comprehensive product feature vector. Finally, it sends the cluster feature labels, user behavior feature vectors, and comprehensive product feature vectors to the model training module.

[0052] The model training module takes the user's user behavior feature vector, cluster feature label, and product comprehensive feature vector as input to the neural collaborative filtering model, and the predicted value of the product interest label as output to train the neural collaborative filtering model, and then sends the neural collaborative filtering model to the interactive display module.

[0053] The interactive display module collects the behavioral feature vectors of the users to be interacted with, as well as the comprehensive feature vectors of the currently displayed products. Based on the behavioral feature vectors and user behavior clusters, it obtains the target cluster label of the users to be interacted with. Based on the behavioral feature vectors, target cluster labels, comprehensive feature vectors, and a neural collaborative filtering model, it generates a detailed display decision.

[0054] An electronic device is proposed, comprising: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0055] The processor executes the aforementioned immersive product information interactive display method by calling the computer program stored in the memory.

[0056] A computer-readable storage medium is proposed, on which an erasable and rewritable computer program is stored;

[0057] When the computer program is run on a computer device, the computer device performs the aforementioned immersive product information interactive display method.

[0058] Compared with the prior art, the beneficial effects of the present invention are:

[0059] This invention collects historical user behavior data from N test users in advance from a virtual device, transforms this data into user behavior feature vectors, and collects visual and textual data of various products. It also collects product interest tags for each test user, clusters the user behavior feature vectors using a clustering algorithm, and divides the test users into K user behavior clusters. The cluster number of each test user serves as the cluster feature label. A computer vision model is used to extract visual feature vectors from the product visual data, and a text analysis model is used to extract semantic feature vectors from the product text data. Finally, the visual and semantic features of each product category are combined to form a comprehensive product feature vector. This approach uses user behavior feature vectors, cluster feature labels, and product comprehensive feature vectors as input to a neural collaborative filtering model, and the predicted values ​​of product interest labels as output. The model is trained to collect behavior feature vectors of users to be interacted with, as well as the comprehensive feature vectors of currently displayed products. Based on the behavior feature vectors and user behavior clusters, the target cluster label for the user to be interacted with is obtained. Based on the behavior feature vectors, target cluster labels, comprehensive feature vectors, and the neural collaborative filtering model, a detailed product display decision is generated. This scheme fully utilizes historical user behavior data and product visual and semantic data to more comprehensively characterize user profiles and product features, providing a rich source of information for personalized recommendations. By dividing users into multiple behavior clusters through clustering algorithms and making recommendations based on the characteristics of the cluster to which the user to be interacted with belongs, the personalization level and accuracy of recommendations are improved. The neural network model can automatically learn the complex mapping relationship between user behavior patterns and product features, which is more advantageous than traditional collaborative filtering algorithms. By analyzing the correlation between behavior patterns and product features, a deeper understanding of influencing factors is achieved, thereby enabling more proactive control and intervention in recommendation strategies. Attached Figure Description

[0060] Figure 1 This is a flowchart of the immersive product information interactive display method in Embodiment 1 of the present invention;

[0061] Figure 2 This is a structural example diagram of the neural collaborative filtering model in Embodiment 1 of the present invention;

[0062] Figure 3 This is a diagram showing the module connection relationships of the immersive product information interactive display system in Embodiment 2 of the present invention;

[0063] Figure 4 This is a schematic diagram of the electronic device structure in Embodiment 3 of the present invention;

[0064] Figure 5 This is a schematic diagram of the computer-readable storage medium structure in Embodiment 4 of the present invention. Detailed Implementation

[0065] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Example 1

[0067] like Figure 1 As shown, the immersive product information interactive display method includes the following steps:

[0068] Step 1: Collect historical user behavior data from N test users in advance from the virtual device, convert the historical user behavior data into user behavior feature vectors, collect visual data and text data of various products, and collect product interest tags for each test user; N is the preset number of test users;

[0069] Step 2: Cluster the user behavior feature vectors using a clustering algorithm to divide the test users into K user behavior clusters, where K is the preset number of user behavior clusters; the cluster number to which each test user belongs is used as the cluster feature label for the test user.

[0070] Step 3: Use a computer vision model to extract visual feature vectors from the product visual data; use a text analysis model to extract semantic feature vectors from the product text data; and combine the visual and semantic features of each product category to form a comprehensive product feature vector.

[0071] Step 4: Use the user's user behavior feature vector, cluster feature label, and product comprehensive feature vector as input to the neural collaborative filtering model, and use the predicted value of the product interest label as output to train the neural collaborative filtering model.

[0072] Step 5: Collect the behavioral feature vectors of the users to be interacted with, as well as the comprehensive feature vectors of the currently displayed products;

[0073] Step 6: Based on behavioral feature vectors and user behavior clusters, obtain the target cluster label of the user to be interacted with; based on behavioral feature vectors, target cluster labels, comprehensive feature vectors, and a neural collaborative filtering model, generate a details display decision;

[0074] The step of collecting historical user behavior data from N test users in advance from the virtual device includes the following steps:

[0075] Step 11: Recruit N volunteers as test users and build a product display space in the virtual reality engine;

[0076] Step 12: Display 3D rendered models of various products for each test user within the product display space, and record the raw behavioral data of each test user in real time through the device's sensors and tracking system; specifically, the raw behavioral data includes, but is not limited to, eye trajectory, hand movements, displacement, etc.; it can be understood that the eye trajectory is collected through the Eye-Tracking module of the head-mounted display, the hand movements are collected through a hand tracking device (such as Leap Motion), and the displacement is obtained by collecting the test user's positioning data;

[0077] Step 13: Use each tester's raw behavioral data as user historical behavioral data;

[0078] In a preferred embodiment, the step of converting user historical behavior data into a user behavior feature vector includes:

[0079] User behavior characteristics of eye trajectory include: the duration and number of times the eye stays on each product's 3D rendering model, the focusing time of the eye on key areas (such as the front of the product), and the speed at which the eye moves on each product's 3D rendering model.

[0080] User behavior characteristics of hand gestures include the frequency of use of various key hand gestures on different products, including but not limited to grasping, placing, and pointing;

[0081] The user behavior characteristics of displacement include the total displacement path length of the test user in the product display space, the distance distribution of the test user when facing different products, and the switching frequency of the test user between scene areas.

[0082] All user behavior features constitute a user behavior feature vector;

[0083] Furthermore, the method for collecting the visual data and textual data of various commodities is as follows:

[0084] Collect images of various products displayed in the product display space as product visual data;

[0085] Collect text descriptions of various products displayed alongside product images in the product display space as product text data; it can be understood that in the virtual product display space, in addition to showing users the visual effects of the product, it also includes showing a brief introduction of the product;

[0086] The product interest tags are entered by each test user after each display of a type of product. The product interest tag is 0 or 1. When the product interest tag is 0, it means that the test user is not interested in the type of product. When the product interest tag is 1, it means that the test user is interested in the type of product.

[0087] Furthermore, the step of using a clustering algorithm to cluster the user behavior feature vectors and dividing the test users into K user behavior clusters includes the following steps:

[0088] Step 21: Treat each user behavior feature vector as an M-dimensional discrete point; where M is the dimension of the user behavior feature vector, and the coordinates of the M-dimensional discrete point in each dimension correspond to a user behavior feature in the user behavior feature vector.

[0089] Step 22: Randomly select K M-dimensional discrete points as initial cluster centers, and the remaining M-dimensional discrete points as non-initial cluster centers;

[0090] Step 23: For each non-initial cluster center, calculate the Euclidean distance to each initial cluster center, and assign each M-dimensional discrete point to the cluster containing the nearest initial cluster center;

[0091] Step 24: Calculate the user behavior features corresponding to each dimension of the M-dimensional discrete points within each cluster, and form a new M-dimensional discrete point by averaging the user behavior features corresponding to each dimension. Use this new M-dimensional discrete point as the new cluster center of the cluster.

[0092] Step 25: Recalculate the Euclidean distance from each M-dimensional discrete point to each new cluster center, and reassign each M-dimensional discrete point to the cluster containing the nearest new cluster center;

[0093] Step 26: Repeat steps 24-25 until the M-dimensional discrete points in all the divided clusters no longer change. Label each cluster with the number k, k = 1, 2, 3, ..., K. Obtain all the M-dimensional discrete points contained in each cluster and form a discrete point set from all the M-dimensional discrete points. The k-th user behavior cluster includes the M-dimensional spatial coordinates of the new cluster center of the k-th cluster and the user behavior feature vector corresponding to each M-dimensional discrete point in the k-th cluster.

[0094] Furthermore, the method for extracting product visual feature vectors from product visual data using a computer vision model is as follows:

[0095] The visual feature vector of a product is extracted from the visual data of the product using a pre-trained graph processing neural network model. The graph processing neural network model can be any one of the CNN model or the ResNet model. It is understood that the visual feature vector of the product can be obtained from the output of the intermediate layer (convolutional layer) of the graph processing neural network model.

[0096] After obtaining the product visual feature vector, principal component analysis is used to project the high-dimensional product visual features to a low dimension, thereby obtaining a product visual feature vector with a lower dimension.

[0097] The method for extracting semantic feature vectors of products using a text analysis model from product text data is as follows:

[0098] The product text data is segmented into word sequences according to word boundaries using a word segmentation tool, such as jieba.

[0099] The bag-of-words model is used to represent each word in the word sequence as a word frequency vector. The weight of each word is calculated based on the inverse document frequency. The Word2Vec neural network is used to map each word to a fixed-dimensional vector space according to its weight. The BERT model is used to transform the set of vector spaces corresponding to the word sequence into text feature vectors based on the context words. These text feature vectors are the product semantic feature vectors.

[0100] Furthermore, the neural collaborative filtering model is trained by using the user's behavior feature vector, cluster feature label, and product comprehensive feature vector as inputs and the predicted value of the product interest label as output.

[0101] The neural collaborative filtering model is constructed as follows:

[0102] like Figure 2 As shown, the neural collaborative filtering model includes an input layer, an embedding layer, an encoding layer, a fully connected layer, and an output layer.

[0103] The input layer includes a first input layer and a second input layer;

[0104] The first input layer is used to receive the user feature vector of the test user, which is composed of user behavior feature vector and clustering feature label;

[0105] The second input layer is used to receive the comprehensive feature vector of the product;

[0106] The embedding layer includes a first embedding layer and a second embedding layer. The first embedding layer is used to map a one-dimensional user feature vector to a low-dimensional dense vector, and the second embedding layer is used to map a product comprehensive feature vector to a corresponding low-dimensional dense vector.

[0107] The encoding layer includes a first encoding layer and a second encoding layer. The first encoding layer encodes the user embedding vector output by the first embedding layer using a multilayer perceptron or a transformer. The second encoding layer encodes the product embedding vector output by the second embedding layer using a multilayer perceptron or a transformer.

[0108] The fully connected layer is used to perform further non-linear encoding on the concatenated vector of the user encoding vector output by the first encoding layer and the product encoding vector output by the second encoding layer;

[0109] The output layer is used to output the predicted values ​​of product interest tags;

[0110] The loss function of the neural collaborative filtering model is -∑ L [y×log(p)+(1-y)×log(1-p)], where y is the product interest label corresponding to each input group, p is the predicted value of the product interest label corresponding to each input group, and L is the total amount of data input into the neural collaborative filtering model;

[0111] The user feature vector and the comprehensive product feature vector of each test user are input into the neural collaborative filtering model, and the neural collaborative filtering model is trained until the loss function converges.

[0112] It is understood that the method of collecting the behavioral feature vector of the user to be interacted with is the same as the method of collecting the user behavioral feature vector;

[0113] The method for obtaining the comprehensive feature vector of the currently displayed product is the same as the method for collecting the comprehensive feature vector of the product, and will not be described again here;

[0114] The method for obtaining the target cluster label of the user to be interacted with, based on behavioral feature vectors and user behavior clusters, is as follows:

[0115] Calculate the Euclidean distance between the behavior feature vector and the center point of each user behavior cluster, and take the user behavior cluster with the closest Euclidean distance as the target cluster, and take the number of the target cluster as the target cluster label;

[0116] The method for generating detail display decisions based on behavioral feature vectors, target cluster labels, comprehensive feature vectors, and a neural collaborative filtering model is as follows:

[0117] The behavioral feature vector and the target cluster label are combined to form the target user feature vector. The target user feature vector and the comprehensive feature vector are then input into the neural collaborative filtering model to obtain the predicted value of the product interest label output by the neural collaborative filtering model.

[0118] Retrieve the display strategy corresponding to the predicted value of the product interest tag from the preset display strategy set, and execute the display strategy; the display strategy is the result of the details display decision.

[0119] The set of display strategies is pre-set by the product display provider. An example of a display strategy is as follows:

[0120] When the predicted value of a product interest tag is 1, more detailed product details, or more detailed text and video descriptions of the product will be displayed for the current product.

[0121] When the predicted value of a product interest tag is 0, skip the currently displayed product and show the next product.

[0122] Example 2

[0123] like Figure 3 As shown, the immersive product information interactive display system includes a training data collection module, a feature vector extraction module, a model training module, and an interactive display module; the modules are connected electrically.

[0124] The training data collection module collects historical user behavior data from N test users in advance from the virtual device, transforms the historical user behavior data into user behavior feature vectors, collects visual data and text data of various products, collects product interest tags for each test user, and sends the user behavior feature vectors, visual data and text data of products to the feature vector extraction module, and sends the product interest tags to the model training module.

[0125] The feature vector extraction module uses a clustering algorithm to cluster user behavior feature vectors, dividing test users into K user behavior clusters. The cluster number of each test user serves as the cluster feature label for the test user. It uses a computer vision model to extract visual feature vectors from product visual data and a text analysis model to extract semantic feature vectors from product text data. The module then combines the visual and semantic features of each product category to form a comprehensive product feature vector. Finally, it sends the cluster feature labels, user behavior feature vectors, and comprehensive product feature vectors to the model training module.

[0126] The model training module takes the user's user behavior feature vector, cluster feature label, and product comprehensive feature vector as input to the neural collaborative filtering model, and the predicted value of the product interest label as output to train the neural collaborative filtering model, and then sends the neural collaborative filtering model to the interactive display module.

[0127] The interactive display module collects the behavioral feature vectors of the users to be interacted with, as well as the comprehensive feature vectors of the currently displayed products. Based on the behavioral feature vectors and user behavior clusters, it obtains the target cluster label of the users to be interacted with. Based on the behavioral feature vectors, target cluster labels, comprehensive feature vectors, and a neural collaborative filtering model, it generates a detailed display decision.

[0128] Example 3

[0129] Figure 4 This is a schematic diagram of an electronic device structure provided in one embodiment of this application. Figure 4 As shown, according to another aspect of this application, an electronic device 100 is also provided. The electronic device 100 may include one or more processors and one or more memories. The memories store computer-readable code, which, when executed by the one or more processors, performs the immersive product information interactive display method described above.

[0130] The method or apparatus according to the embodiments of this application can also be used by means of Figure 4 The architecture of the electronic device shown is used to implement this. For example... Figure 4 As shown, the electronic device 100 may include a bus 101, one or more CPUs 102, ROM 103, RAM 104, a communication port 105 connected to a network, an input / output component 106, a hard disk 107, etc. The storage device in the electronic device 100, such as ROM 103 or hard disk 107, may store the immersive product information interactive display method provided in this application. The immersive product information interactive display may, for example, include the following steps:

[0131] Furthermore, the electronic device 100 may also include a user interface 108. Of course, Figure 4 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 4 One or more components in the illustrated electronic device.

[0132] Example 4

[0133] Figure 5 This is a schematic diagram of a computer-readable storage medium structure provided in one embodiment of this application. Figure 5The diagram illustrates a computer-readable storage medium 200 according to one embodiment of this application. The computer-readable storage medium 200 stores computer-readable instructions. When executed by a processor, the computer-readable instructions can perform the immersive product information interactive display method according to an embodiment of this application, as described above with reference to the accompanying drawings. The computer-readable storage medium 200 includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0134] Furthermore, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, this application provides a non-transitory machine-readable storage medium storing machine-readable instructions that can be executed by a processor to perform instructions corresponding to the method steps provided in this application. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the method of this application.

[0135] The methods, apparatus, and devices of this application may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the method is for illustrative purposes only, and the steps of the method of this application are not limited to the order specifically described above, unless otherwise specifically stated. Furthermore, in some embodiments, this application may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the method according to this application. Thus, this application also covers recording media storing programs for performing the method according to this application.

[0136] In addition, the parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of the corresponding technical solutions in the prior art have not been described in detail, so as to avoid excessive elaboration.

[0137] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0138] The preset parameters or preset thresholds mentioned above are all set by those skilled in the art based on actual conditions or obtained through large-scale data simulation.

[0139] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. An immersive product information interactive display method, characterized in that, Includes the following steps: Step 1: Collect historical user behavior data from N test users in advance from the virtual device, convert the historical user behavior data into user behavior feature vectors, collect visual data and text data of various products, and collect product interest tags for each test user; N is the preset number of test users; Step 2: Cluster the user behavior feature vectors using a clustering algorithm to divide the test users into K user behavior clusters, where K is the preset number of user behavior clusters; the cluster number to which each test user belongs is used as the cluster feature label for the test user. Step 3: Use a computer vision model to extract visual feature vectors from the product visual data; use a text analysis model to extract semantic feature vectors from the product text data; and combine the visual and semantic features of each product category to form a comprehensive product feature vector. Step 4: Use the user's user behavior feature vector, cluster feature label, and product comprehensive feature vector as input to the neural collaborative filtering model, and use the predicted value of the product interest label as output to train the neural collaborative filtering model. Step 5: Collect the behavioral feature vectors of the users to be interacted with, as well as the comprehensive feature vectors of the currently displayed products; Step 6: Based on behavioral feature vectors and user behavior clusters, obtain the target cluster labels for the users to be interacted with; Based on behavioral feature vectors, target cluster labels, comprehensive feature vectors, and a neural collaborative filtering model, a detailed display decision is generated.

2. The immersive product information interactive display method according to claim 1, characterized in that, The process of pre-collecting historical user behavior data from N test users from virtual devices includes the following steps: Step 11: Recruit N volunteers as test users and build a product display space in the virtual reality engine; Step 12: Display 3D rendered models of various products for each test user in the product display space, and record the raw behavioral data of each test user in real time through the device's sensors and tracking system; Step 13: Use each tester's raw behavioral data as user historical behavioral data.

3. The immersive product information interactive display method according to claim 2, characterized in that, The step of using a clustering algorithm to cluster user behavior feature vectors and dividing test users into K user behavior clusters includes the following steps: Step 21: Treat each user behavior feature vector as an M-dimensional discrete point; where M is the dimension of the user behavior feature vector, and the coordinates of the M-dimensional discrete point in each dimension correspond to a user behavior feature in the user behavior feature vector. Step 22: Randomly select K M-dimensional discrete points as initial cluster centers, and the remaining M-dimensional discrete points as non-initial cluster centers; Step 23: For each non-initial cluster center, calculate the Euclidean distance to each initial cluster center, and assign each M-dimensional discrete point to the cluster containing the nearest initial cluster center; Step 24: Calculate the user behavior features corresponding to each dimension of the M-dimensional discrete points within each cluster, and form a new M-dimensional discrete point by averaging the user behavior features corresponding to each dimension. Use this new M-dimensional discrete point as the new cluster center of the cluster. Step 25: Recalculate the Euclidean distance from each M-dimensional discrete point to each new cluster center, and reassign each M-dimensional discrete point to the cluster containing the nearest new cluster center; Step 26: Repeat steps 24-25 until the M-dimensional discrete points in all the divided clusters no longer change. Label each cluster with the number k, k = 1, 2, 3, ..., K. Obtain all the M-dimensional discrete points contained in each cluster and form a discrete point set from all the M-dimensional discrete points. The k-th user behavior cluster includes the M-dimensional spatial coordinates of the new cluster center of the k-th cluster and the user behavior feature vector corresponding to each M-dimensional discrete point in the k-th cluster.

4. The immersive product information interactive display method according to claim 3, characterized in that, The method for extracting semantic feature vectors of products using a text analysis model from product text data is as follows: The product text data is segmented into word sequences according to word boundaries using a word segmentation tool. The bag-of-words model is used to represent each word in the word sequence as a word frequency vector. The weight of each word is calculated based on the inverse document frequency. The Word2Vec neural network is used to map each word to a fixed-dimensional vector space according to its weight. The BERT model is used to transform the set of vector spaces corresponding to the word sequence into text feature vectors based on the context words. These text feature vectors are the semantic feature vectors of the product.

5. The immersive product information interactive display method according to claim 4, characterized in that, The method for training the neural collaborative filtering model is as follows: Construct a neural collaborative filtering model; The user feature vector and the comprehensive product feature vector of each test user are input into the neural collaborative filtering model, and the neural collaborative filtering model is trained until the loss function converges.

6. The immersive product information interactive display method according to claim 5, characterized in that, The neural collaborative filtering model includes an input layer, an embedding layer, an encoding layer, a fully connected layer, and an output layer. The input layer includes a first input layer and a second input layer; The first input layer is used to receive the user feature vector of the test user, which is composed of user behavior feature vector and clustering feature label; The second input layer is used to receive the comprehensive feature vector of the product; The embedding layer includes a first embedding layer and a second embedding layer. The first embedding layer is used to map a one-dimensional user feature vector to a low-dimensional dense vector, and the second embedding layer is used to map a product comprehensive feature vector to a corresponding low-dimensional dense vector. The encoding layer includes a first encoding layer and a second encoding layer. The first encoding layer encodes the user embedding vector output by the first embedding layer using a multilayer perceptron or a transformer. The second encoding layer encodes the product embedding vector output by the second embedding layer using a multilayer perceptron or a transformer. The fully connected layer is used to perform further non-linear encoding on the concatenated vector of the user encoding vector output by the first encoding layer and the product encoding vector output by the second encoding layer; The output layer is used to output the predicted values ​​of product interest tags; The loss function of the neural collaborative filtering model is -∑ L [y×log(p)+(1-y)×log(1-p)], where y is the product interest tag corresponding to each input group, p is the predicted value of the product interest tag corresponding to each input group, and L is the total amount of data input into the neural collaborative filtering model.

7. The immersive product information interactive display method according to claim 6, characterized in that, The method for generating detail display decisions based on behavioral feature vectors, target cluster labels, comprehensive feature vectors, and a neural collaborative filtering model is as follows: The behavioral feature vector and the target cluster label are combined to form the target user feature vector. The target user feature vector and the comprehensive feature vector are then input into the neural collaborative filtering model to obtain the predicted value of the product interest label output by the neural collaborative filtering model. Retrieve the display strategy corresponding to the predicted value of the product interest tag from the preset display strategy set, and execute the display strategy; the display strategy is the result of the details display decision.

8. An immersive product information interactive display system, used to implement the immersive product information interactive display method according to any one of claims 1-7, characterized in that, It includes a training data collection module, a feature vector extraction module, a model training module, and an interactive display module; the modules are connected electrically. The training data collection module collects historical user behavior data from N test users in advance from the virtual device, transforms the historical user behavior data into user behavior feature vectors, collects visual data and text data of various products, collects product interest tags for each test user, and sends the user behavior feature vectors, visual data and text data of products to the feature vector extraction module, and sends the product interest tags to the model training module. The feature vector extraction module uses a clustering algorithm to cluster user behavior feature vectors, dividing test users into K user behavior clusters. The cluster number of each test user serves as the cluster feature label for the test user. It uses a computer vision model to extract visual feature vectors from product visual data and a text analysis model to extract semantic feature vectors from product text data. The module then combines the visual and semantic features of each product category to form a comprehensive product feature vector. Finally, it sends the cluster feature labels, user behavior feature vectors, and comprehensive product feature vectors to the model training module. The model training module takes the user's user behavior feature vector, cluster feature label, and product comprehensive feature vector as input to the neural collaborative filtering model, and the predicted value of the product interest label as output to train the neural collaborative filtering model, and then sends the neural collaborative filtering model to the interactive display module. The interactive display module collects the behavioral feature vectors of the users to be interacted with, as well as the comprehensive feature vectors of the currently displayed products. Based on the behavioral feature vectors and user behavior clusters, it obtains the target cluster label of the users to be interacted with. Based on the behavioral feature vectors, target cluster labels, comprehensive feature vectors, and a neural collaborative filtering model, it generates a detailed display decision.

9. An electronic device, characterized in that, include: Processor and memory, of which, The memory stores computer programs that can be called by the processor; The processor executes the immersive product information interactive display method according to any one of claims 1-7 in the background by calling the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that, It contains erasable and rewritable computer programs; When the computer program is run on a computer device, the computer device executes the immersive product information interactive display method according to any one of claims 1-7 in the background.

Images