Multi-modal industrial product recommendation method and system based on mixed attention mechanism

Abstract

The invention provides a multi-modal industrial product recommendation method and system based on a mixed attention mechanism, and relates to the field of intelligent recommendation. The method comprises the following steps: acquiring multi-modal product data comprising text data and image data, and constructing a multi-modal industrial product knowledge graph comprising a product information triple, a text triple, an image triple and a user product interaction quadruple based on the multi-modal product data; extracting user behavior sequence features in a user product interaction tetrad by using one-hot coding, and obtaining user short-term interest features by adopting a time-sensitive attention mechanism; extracting and embedding entity node features of the multi-modal industrial product knowledge graph, and inputting a multi-head attention mechanism to extract industrial product features; and fusing the short-term interest features of the user and the industrial product features to obtain comprehensive preference features of the user. According to the method, the multi-modal data is processed through the mixed attention mechanism, the long and short preferences of the users are fused, heterogeneous information can be effectively integrated, and the industrial product recommendation accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent recommendation technology, and in particular to a multimodal industrial product recommendation method and system based on a hybrid attention mechanism. Background Art

[0002] With the development of industrial Internet and intelligent manufacturing, data related to industrial products have shown multimodal characteristics. On the one hand, these data cover a variety of forms, including not only text descriptions of products, but also images of products. On the other hand, the scale of data is growing exponentially, user behavior is showing an increasingly diversified trend, and user choices are affected by a combination of factors.

[0003] In practical applications, using recommendation algorithms to recommend information of interest to users has become an increasingly important technical means. However, due to the diversity of user behaviors, directly using multimodal information for recommendation faces problems such as information heterogeneity. Existing technologies cannot effectively integrate these heterogeneous information and construct a unified representation. Although some advanced deep learning models can handle the relationship between modalities to a certain extent, they are still insufficient in maximizing the potential correlation and complementarity between different modalities while maintaining information integrity.

[0004] At the same time, in the field of industrial products, users' requirements for recommendations are not just to find relevant products, but also to have a deep understanding of whether the products really meet their complex needs, including production process requirements, quality standards, cost-effectiveness, etc. When faced with complex and changing user interests, current recommendation methods still have limitations, especially in terms of capturing and understanding user detailed preferences and information fusion. They fail to fully consider users' short-term preferences and long-term overall preferences. The recommendation results are too dependent on historical behavior and ignore changes in user interests, resulting in low accuracy of recommendation results. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a multimodal industrial product recommendation method and system based on a hybrid attention mechanism, which utilizes a hybrid attention mechanism that integrates a time-sensitive attention mechanism and a multi-head attention mechanism, and makes full use of multimodal information to fully model user preferences and improve recommendation performance.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a multimodal industrial product recommendation method based on a hybrid attention mechanism, comprising: Acquiring multimodal product data for preprocessing, wherein the multimodal product data includes text data and image data; Constructing a multimodal industrial product knowledge graph based on the preprocessed multimodal product data, wherein the multimodal industrial product knowledge graph includes a product information triple, a text triple, an image triple, and a user-product interaction quadruple; One-hot encoding is used to extract user behavior sequence features in user-product interaction quadruple. Based on CLIP model and ConvE model, entity node feature embedding of multimodal industrial product knowledge graph is extracted. Based on the user behavior sequence characteristics, the time-sensitive attention mechanism is used to obtain the user's short-term interest characteristics; based on the entity node feature embedding, the multi-head attention mechanism is used to extract the industrial product characteristics; By integrating the user's short-term interest characteristics with the industrial product characteristics, we can obtain the user's comprehensive preference characteristics. We make recommendations based on the user's comprehensive preference characteristics and score the user's comprehensive preference characteristics.

[0007] Preferably, the step of constructing a multimodal industrial product knowledge graph based on the preprocessed multimodal product data specifically includes: Obtain preprocessed text data and image data, extract product ID, relationship and attribute to construct product knowledge graph triples, extract product ID, description and text information to construct text triples, extract product ID, image and image information to construct image triples, extract user ID, product ID, rating and timestamp to construct user-product interaction quadruples.

[0008] Preferably, the extracting user behavior sequence features in the user-product interaction quadruple by using one-hot encoding specifically includes: Extract the user ID, the user-product interaction behavior and its corresponding timestamp from the user-product interaction quadruple, and obtain preliminary user features through one-hot encoding; Input the preliminary user features into the linear graph convolution module and the attention mechanism layer to obtain the user's product interaction sequence; The user behavior sequence features are obtained based on the user's product interaction sequence, interaction timestamp and product ID.

[0009] Preferably, the method of extracting entity node feature embedding of the multimodal industrial product knowledge graph based on the CLIP model and the ConvE model specifically includes: Extract text feature embedding of text triples and image feature embedding of image triples based on CLIP model; The ConvE model is used to extract user nodes, product nodes and attribute nodes from the user-product interaction quadruple and the product knowledge graph triple to obtain the initial entity features. Based on the initial entity features, the initial product entity feature embedding is obtained. The entity node feature embedding is obtained based on the text feature embedding, image feature embedding and initialized product entity feature embedding.

[0010] Preferably, the method adopts a time-sensitive attention mechanism based on user behavior sequence characteristics to obtain user short-term interest characteristics, specifically including: Extract the interaction time information from the user-product interaction quadruple and calculate the user's time interval characteristics; Construct the intermediate vector of the time-sensitive attention mechanism based on the time interval features and user behavior sequence features; Combine the preliminary user features with the intermediate vector of the time-sensitive attention mechanism and normalize them to obtain the normalized attention weights; The normalized attention weight and the user behavior sequence characteristics are weighted summed to obtain the user's short-term interest characteristics.

[0011] Preferably, the entity node feature embedding adopts a multi-head attention mechanism to extract industrial product features, specifically including: Calculate the attention score based on the entity node feature embedding; Based on the attention score, the adjacent embeddings of the head entity are summed to obtain the first-order structural information; The double interactive aggregation function is used to concatenate the structural information of all heads, and the head entity and its first-order structural information are aggregated. By stacking multiple attention layers, high-order structural information is captured to obtain industrial product features.

[0012] Preferably, the scoring of the user's comprehensive preference characteristics specifically includes:

[0013] in, represents the comprehensive preference characteristics of users after fusion, The final representation of industrial product features after splicing different layers is shown. Indicates transpose.

[0014] In a second aspect, the present invention provides a multimodal industrial product recommendation system based on a hybrid attention mechanism, comprising: A data processing layer, used to obtain multimodal product data for preprocessing, wherein the multimodal product data includes text data and image data; A knowledge graph layer, used to construct a multimodal industrial product knowledge graph based on the preprocessed multimodal product data, wherein the multimodal industrial product knowledge graph includes a product information triple, a text triple, an image triple, and a user-product interaction quadruple; The knowledge feature extraction layer is used to extract the user behavior sequence features in the user-product interaction quadruple using one-hot encoding; based on the CLIP model and the ConvE model, it extracts the entity node feature embedding of the multimodal industrial product knowledge graph; The attention mechanism layer is used to use the time-sensitive attention mechanism based on the user behavior sequence characteristics to obtain the user's short-term interest characteristics; based on the entity node feature embedding, a multi-head attention mechanism is used to extract industrial product features; The intelligent recommendation and application evaluation layer is used to integrate the user's short-term interest characteristics with the industrial product characteristics to obtain the user's comprehensive preference characteristics; make recommendations based on the user's comprehensive preference characteristics, and score the user's comprehensive preference characteristics.

[0015] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in a multimodal industrial product recommendation method based on a hybrid attention mechanism described in the first aspect.

[0016] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a multimodal industrial product recommendation method based on a hybrid attention mechanism described in the first aspect are implemented.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention introduces multimodal information as auxiliary information to construct a multimodal industrial product knowledge graph to integrate different types of data, thereby considering more dimensions of information in user behavior interest modeling and better capturing user preferences. The multimodal industrial product knowledge graph not only contains triple knowledge around product information, which can obtain rich and complete project product information; on this basis, it also includes a user-product interaction quadruple that considers user and product interaction information, in which the "rating" element is used as a quantitative representation of the interaction behavior to accurately measure the user's preference for the product and demand tendency. At the same time, the timestamp records the time when the user queries the product, introduces the time characteristics between the user and the product, and thus realizes the tracking and analysis of the dynamic changes in user interests. The construction of a multimodal industrial product knowledge graph effectively improves the accuracy and timeliness of recommendations, and can fully meet the complex and changing interest needs of users.

[0018] (2) The present invention proposes a hybrid attention mechanism that combines a time-sensitive attention mechanism and a multi-head attention mechanism. The time-sensitive attention mechanism is used to effectively capture the user's short-term interest preferences containing time information by weighting recent interactive behaviors. At the same time, the multi-head attention mechanism is used to mine the user's preferences from multiple dimensions by dividing the input information into multiple heads, thereby comprehensively capturing the user's overall interests. The synergy of these two mechanisms can obtain the user's comprehensive preferences, thereby deeply analyzing the complex interactive relationship between the user and the industrial product project, and providing a rich representation for the calculation of the recommendation results. This helps to improve the fit between the recommendation system and the actual needs of users, making the recommendation results more accurate and effective. It can not only allow users to have a more satisfactory experience in the product selection process, but also help industrial products to achieve more efficient supply and demand docking in the market circulation link, and promote the development of industrial products in the market.

[0019] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their description are used to explain the present invention but do not constitute a limitation of the present invention.

[0021] Figure 1 A main flow chart of a multimodal industrial product recommendation method based on a hybrid attention mechanism provided by an embodiment of the present invention; Figure 2 A detailed flow chart of a multimodal industrial product recommendation method based on a hybrid attention mechanism provided by an embodiment of the present invention; Figure 3 A flowchart of text and image feature extraction provided by an embodiment of the present invention; Figure 4 A flow chart of a time-sensitive attention mechanism provided for an embodiment of the present invention; Figure 5 Flowchart of the multi-head attention mechanism provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0023] Embodiment 1 like Figure 1 As shown, this embodiment discloses a multimodal industrial product recommendation method based on a hybrid attention mechanism, comprising the following steps: S1: obtaining multimodal product data for preprocessing, wherein the multimodal product data includes text data and image data; S2: construct a multimodal industrial product knowledge graph based on the preprocessed multimodal product data, wherein the multimodal industrial product knowledge graph includes product information triples, text triples, image triples, and user-product interaction quadruples; S3: Use one-hot encoding to extract user behavior sequence features in user-product interaction quadruple; extract entity node feature embedding of multimodal industrial product knowledge graph based on CLIP model and ConvE model; S4: Based on the user behavior sequence characteristics, the time-sensitive attention mechanism is used to obtain the user's short-term interest characteristics; based on the entity node feature embedding, the multi-head attention mechanism is used to extract the industrial product characteristics; S5: Integrate user short-term interest characteristics with industrial product characteristics to obtain user comprehensive preference characteristics; make recommendations based on user comprehensive preference characteristics and score user comprehensive preference characteristics.

[0024] Next, combine Figure 2 , a multimodal industrial product recommendation method based on a hybrid attention mechanism disclosed in this embodiment is described in detail.

[0025] In S1, the raw data is preprocessed to make the data more standardized, concise and easy to analyze.

[0026] (1) Data collection Obtain a multimodal industrial product recommendation dataset containing text and image information from an open recommendation data source, download the selected industrial dataset through a data platform or open data warehouse, decompress the data, and extract the original data.

[0027] (2) Data preprocessing 1) Data cleaning After obtaining the original data set, the data is cleaned to ensure the quality and consistency of the data. First, the data in the industrial product data set needs to be filtered to remove records with missing text information or image information and duplicate records, because these incomplete samples cannot provide effective support for subsequent multimodal analysis. Secondly, further screening is performed to filter according to the frequency of interaction between users and industrial products, and data information with more than 20 text interactions is screened out to facilitate the division of the data set.

[0028] 2) Text processing In the process of text data processing, entries containing garbled characters, unrecognizable characters or irregular formats must be excluded to ensure the quality and consistency of the data. For example, text containing garbled characters, unrecognizable characters or incorrect formats.

[0029] 3) Image processing During the image processing process, in order to ensure the quality and effectiveness of the images used in the product, it is necessary to screen out images with higher resolution and clarity from all available images as the image resources for final use.

[0030] In S2, based on the processed data, the data information is mined, and knowledge tuples are extracted by identifying entities, relationships and their attributes in the data, thereby constructing a multimodal industrial product knowledge graph. At the same time, in order to simplify data processing in the knowledge graph and improve computing efficiency, the knowledge graph data is mapped and digital encoding is used instead of string representation, which can take up less storage space.

[0031] (1) Construction of multimodal industrial product knowledge graph When faced with processed data sets, in order to achieve efficient data utilization and deep mining, knowledge tuples can be extracted, classified and structured according to different data types.

[0032] Specifically, “product ID, relationship and attribute” are extracted to construct product knowledge graph triples to accurately represent various relationships between industrial products and their related attribute information.

[0033] In order to enrich the representation of industrial product items, in addition to the attribute information related to industrial products, other product descriptions and product image information are also used to construct text triples and image triples.

[0034] Specifically, "product ID, description and text information" are extracted to construct text triples to represent industrial products and their related descriptive text information, and better realize the structural and semantic processing of text data. "Product ID, image and image information" are extracted to construct image triples to extract visual content related to industrial products and convert them into processable image data.

[0035] In order to accurately grasp the users' complex needs for industrial products, gain in-depth insights into users' interest preferences at different times, and comprehensively reflect the interaction details between users and products, we extract "user ID, product ID, rating, and timestamp" to construct a user-product interaction quadruple to capture users' evaluation of industrial products and their interaction time, thereby providing support for applications such as recommendation systems.

[0036] The four-tuple proposed in this embodiment uses "ratings" to quantify the user's preference for the product and demand tendency, and accurately grasps the user's complex needs. By recording the user's query time through timestamps and introducing time characteristics, it can track and analyze the dynamic changes of user interests and gain in-depth insights into interest preferences in different periods. It can more comprehensively reflect the details of the interaction between users and products, and provide strong support for the recommendation system.

[0037] Through the above-mentioned form of multi-group structured representation, multimodal data can be effectively integrated and the knowledge expression and reasoning capabilities of the system can be improved.

[0038] (2) Multi-tuple data mapping In the process of processing knowledge tuples, by analyzing various types of information in the tuple, each independent element is mapped to a unique integer identifier, thereby converting the original symbolic or textual information into a digital representation. Using digital representation can simplify data processing and calculation, and take up less storage space, making it easier to perform vector processing and improve computing efficiency.

[0039] A variety of feature extraction algorithms are used to comprehensively extract various feature information in the knowledge graph.

[0040] (1) User behavior sequence characteristics In order to extract the interaction features between users and products in the user-product interaction quadruple, the user ID, the interaction behavior between users and products and their corresponding timestamps in the user-product interaction quadruple are extracted, and the unique hot encoding is input to obtain the feature information of the interaction between users and industrial products, that is, the preliminary user features .

[0041] It should be understood that the interaction between the user and the product can be indirectly obtained by analyzing and processing the user-product interaction quadruple. For example, the score can be regarded as a quantitative result of an interaction behavior, and different score values ​​can represent different degrees of interaction (such as a high score indicates that the user likes the product and actively interacts with it, and a low score indicates dissatisfaction, etc.); the timestamp can reflect the time sequence and frequency of the interaction behavior. By analyzing the distribution and interval of the timestamp, the behavioral characteristics such as the activeness and periodicity of the user's interaction with the product can be inferred. Those skilled in the art can define the interaction between the user and the product according to the actual situation.

[0042] These preliminary user characteristics It is passed as input to the linear graph convolution module, where the embedding of users and industrial products is optimized using linear aggregation and index lookup operations to generate preliminary embedding representations of users and industrial product items. Subsequently, the embedded feature representation is further enriched and strengthened through the attention mechanism layer to capture more detailed and complex user-industrial product interaction relationships, and then the user behavior sequence features are obtained based on the user's product interaction sequence, interaction timestamp, and product ID. .

[0043] (2) Entity node feature embedding In order to enhance the feature mining of multimodal data, the CLIP model is used to extract features from text triples and image triples in the multimodal industrial product knowledge graph. Figure 3As shown in Figure 2, CLIP can effectively align text and images in a shared feature space through joint training, thereby extracting the semantic features of the text. and the visual features of the image .

[0044] In the feature extraction process of product knowledge graph triples, the ConvE method is used to conduct in-depth analysis of user-product interaction quadruples and product knowledge graph triples to extract semantic features of entities and relationships. In this part, the user nodes, product nodes, and attribute nodes in the multi-tuple are fused into entity nodes that contain all nodes. When using the ConvE method to extract entity node features, the relevant entity features must be initialized first. And initialize the relationship characteristics , and then fuse the text features, image features, and product entity features in the initialized original entity features. The fusion method is as follows:

[0045] in, represents the original entity feature embedding, represents the product entity feature embedding in the original entity embedding, represents image feature embedding, Represents text feature embedding, concat() represents concatenation operation, Represents the fused entity embedding.

[0046] After obtaining the fused entity embedding, convolution operation is performed through two convolution layers to extract local features. The feature map after the convolution operation is flattened into a one-dimensional vector through the view operation, ready to be input into the fully connected layer for further processing. According to different aggregation methods, the adjacent node information of the graph and the features extracted by convolution are combined and fused through addition. Finally, the fused features are input into the fully connected layer for output, and all node features are output. , including the header entity and tail entity .

[0047] The attention mechanism layer mainly includes two types of attention mechanisms. One is the time-sensitive attention mechanism to extract the user's short-term interests, and the other is the multi-head attention mechanism to comprehensively focus on the user's overall interests from multiple aspects.

[0048] In S3, a time-sensitive attention mechanism is used to weight user-product interactions containing time information to dynamically adjust the weight of user interests and thus extract their short-term interest representations.

[0049] (1) Time-Sensitive Attention Mechanism like Figure 4As shown, the time-sensitive attention mechanism effectively identifies users' preferences for product items within a specific time period by considering users' historical behaviors and time factors, analyzing time intervals and the interaction history between users and industrial products.

[0050] 1) Using the interaction time information in the user-product interaction quadruple, calculate the user's time interval characteristics. The calculation formula is:

[0051] in, and denote weight and bias parameters respectively, represents the tanh activation function, and Represents the timestamp and time interval features of users at different time steps Represents the encoding time step and time step The absolute time distance between them.

[0052] 2) Use the time interval features and user behavior sequence features to construct the intermediate vector of the time-sensitive attention mechanism. The calculation formula is:

[0053] in, is the user behavior sequence feature, indicating that the time step in the user u behavior sequence is The vector representation of the product of Represents the time interval characteristics, and represents the learnable weight matrix, Represents the bias vector.

[0054] 3) Use the user's embedded representation to associate with the intermediate vector of the time-sensitive attention mechanism and use the normalization function to normalize it to obtain the normalized attention weight. The calculation formula is:

[0055] in, is the preliminary user feature, which represents the embedded representation of the user; The intermediate vector representing the time-sensitive attention mechanism: represents transpose; Indicates that user u is The intermediate vector obtained at .

[0056] 4) After obtaining the attention weight, the attention weight is used to perform weighted summation with the user behavior sequence features to obtain the feature representation of the user's short-term interest. The calculation formula is:

[0057] This embodiment uses the interaction time information to calculate the time interval feature, which can accurately analyze the user's behavior changes at different time points and effectively identify preferences within a specific time period; further, the intermediate vector is constructed by combining the time interval feature with the user behavior sequence feature, and then the attention weight is obtained through correlation and normalization, which can highlight the importance of recent interactions. Finally, the short-term interest feature representation is obtained by weighted summation of the attention weight and the user behavior sequence feature, which can timely reflect the user's current interest dynamics, avoid excessive reliance on historical behavior in recommendation results, make recommendations more in line with the user's current needs, and improve the timeliness and accuracy of recommendations.

[0058] In S4, the multi-head attention mechanism is used to utilize multiple attention heads to comprehensively focus on the user's overall interest representation and the multimodal feature representation of industrial products from multiple perspectives.

[0059] like Figure 5 As shown in the figure, the multi-head attention mechanism aims to capture the multi-level features and complex relationships in the input data from different perspectives by operating multiple attention heads in parallel. Each attention head independently calculates the attention weight and generates the corresponding feature representation according to its own focus. Finally, by integrating the outputs of each attention head, the model obtains a richer and more diverse representation capability.

[0060] 1) First, use ConvE to extract multimodal features to obtain the comprehensive features of the node and relationship characteristics .

[0061] 2) Using the entity and relationship embeddings extracted from the multimodal industrial product knowledge graph and the user-product interaction knowledge graph through relevant operations, the calculation formula for the attention score is obtained as follows:

[0062] in, is the weight matrix of the kth head, is the relational transformation matrix, , and They respectively represent the embedding vectors of the fusion of the head entity, relationship and tail entity in the user-product interaction quadruple, the product knowledge graph triple, the text triple and the image triple.

[0063] 3) Collect the first-order structural information by summing the embeddings of the neighboring entities of the head entity h (i.e., the tail entity corresponding to the head entity). The calculation formula is:

[0064] in, is the neighbor set of the head entity h.

[0065] 4) Use the double interactive aggregation function to concatenate the structural information of all heads and perform aggregation operations on the head entity and its first-order neighbor information. The calculation formula is:

[0066] in, and represents the learnable transformation matrix, is the element product, and concat represents the concatenation operation.

[0067] 5) By stacking multiple attention layers to capture high-order structural information and obtain more supervision signals from distant nodes, the entity representation can be obtained by recursive calculation, and the calculation formula is:

[0068] in, All entity feature representations obtained by the multi-head attention mechanism, including user entity feature representation and product entity feature representation .

[0069] This embodiment can capture the multi-level features and complex relationships of the input data from different angles by operating multiple attention heads in parallel, so that the model has richer and more diverse representation capabilities. Secondly, each attention head independently calculates the attention weight and generates the corresponding feature representation, and then integrates the output, which can fully focus on the overall interests of users and the multimodal features of products. In addition, by collecting first-order structural information by summing the embeddings of adjacent entities of the head entity, and using the dual interaction aggregation function for aggregation operations, it can also capture high-order structural information and obtain more supervisory signals from distant nodes, thereby improving the model's understanding and processing capabilities of user-product interaction information and optimizing the recommendation effect.

[0070] In S5, by comprehensively utilizing the user's comprehensive interest representation and the multimodal feature representation of industrial products, the inner product between the user's interest vector and the industrial product item feature vector is calculated, and the similarity between the two is evaluated, so that accurate product recommendations can be provided based on the user's interest preferences.

[0071] (1) Multi-layer representation fusion The representations of different layers obtained through recursion can capture node features and connection information to different degrees. Then, the layer aggregation mechanism is used to splice the representations of users and industrial products at each layer, and the information of different layers is integrated into a unified embedding to more comprehensively represent users and industrial product items. The calculation formula is:

[0072]

[0073] Where L is the number of polymerization layers, and is the feature representation of each layer of users and products, and concat represents the concatenation operation.

[0074] (2) User interest feature fusion The user's comprehensive preference feature representation is obtained by fusing the user's short-term interest preference features in the time-sensitive attention mechanism with the user's overall interest preference features obtained by the multi-head attention mechanism. The calculation formula is:

[0075] in, Represents the overall interest representation of the user after concatenation of different layers, represents the user's short-term interest feature representation, Represents the comprehensive user preference representation after fusion.

[0076] This solution adopts a hybrid attention mechanism. The time-sensitive attention mechanism can capture users' short-term interests and use interaction time information to accurately analyze behavior changes, avoiding over-reliance on historical behavior; the multi-head attention mechanism focuses on overall interests and multimodal features from multiple angles, enriches representation capabilities, and captures high-order structural information. By combining the features obtained by the two attention mechanisms through user interest feature fusion, it can not only reflect current interests in a timely manner, but also fully understand user needs, providing strong support for providing recommendations with high accuracy and timeliness, and improving recommendation effects and user experience.

[0077] (3) Calculate the prediction score The matching score between the user and the industrial product is predicted by calculating the inner product of their final representations. The calculation formula is:

[0078] in, represents the comprehensive preference representation of users after fusion, It represents the final representation of the industrial product project after splicing different layers.

[0079] Furthermore, the performance of the method is comprehensively evaluated through a series of evaluation indicators, and the user preference modeling and recommendation performance of the method are evaluated through comparative analysis with traditional recommendation algorithms.

[0080] In the development and application of recommendation systems, evaluation indicators are the core tools for measuring the performance of model methods. Through reasonable evaluation indicators, we can fully understand the performance of model methods in different dimensions, and then objectively evaluate their effects. The following are the evaluation indicators used in the method: (1) Precision Precision rate refers to the ratio of the number of products that are accurately recommended (i.e., the user is actually interested in or has actually interacted with) in the recommendation list to the total number of products in the recommendation list. The calculation formula is:

[0081] in, represents the number of users in the user set U, k represents the first k recommendation results considered when calculating the precision rate, is a two-dimensional array, Indicates whether the i-th product in the recommendation list of user u is a positive sample.

[0082] (2) Recall The recall rate is a measure of the ratio of the number of products that the recommendation system can successfully recall (i.e. recommend) that the user is actually interested in or has actually interacted with to the number of all relevant products. The calculation formula is:

[0083] in, represents the number of users in the user set U, k represents the first k recommendation results considered when calculating the precision rate, is a two-dimensional array, Indicates whether the i-th product in the recommendation list of user u is a positive sample, Represents the number of real related products for user u.

[0084] (3) Normalized Discounted Cumulative Gain NDCG Normalized discounted cumulative gain is an evaluation metric used to measure the performance of recommendation systems or information retrieval systems. Its main purpose is to comprehensively evaluate the relevance and ranking quality of recommendation results.

[0085]

[0086] Among them, the cumulative loss gain The calculation formula is:

[0087] in, represents the number of users in the user set U, k represents the first k recommendation results considered when calculating the precision rate, is an indicator function, which is 1 when the i-th product in the recommendation list is in the real relevant product set R(u) of user u, and 0 otherwise.

[0088] Ideal loss cumulative gain The calculation formula is:

[0089] in, Indicates taking The smaller of the two values ​​of and k.

[0090] After preprocessing the multimodal product data, this specific embodiment extracts multi-tuples, uses unique hot encoding and two attention mechanisms to obtain user short-term interest features and industrial product features respectively, and then fuses them to obtain user comprehensive preference features and scores. In this process, by processing heterogeneous information, it is possible to fully explore the potential correlation and complementarity between different modalities while maintaining information integrity. In terms of industrial product recommendations, not only product relevance is considered, but also the complex needs of users in production processes, quality standards, cost-effectiveness, etc. At the same time, by integrating users' short-term interest features and long-term overall preferences, the recommendation results can avoid excessive reliance on historical behaviors, effectively capture changes in user interests, and overcome the shortcomings of existing recommendation methods in capturing user refined preferences and information fusion, thereby greatly improving the accuracy and practicality of industrial product recommendations.

[0091] Embodiment 2 This embodiment provides a multimodal industrial product recommendation system based on a hybrid attention mechanism, including: A data processing layer, used to obtain multimodal product data for preprocessing, wherein the multimodal product data includes text data and image data; A knowledge graph layer, used to construct a multimodal industrial product knowledge graph based on the preprocessed multimodal product data, wherein the multimodal industrial product knowledge graph includes a product information triple, a text triple, an image triple, and a user-product interaction quadruple; The knowledge feature extraction layer is used to extract the user behavior sequence features in the user-product interaction quadruple using one-hot encoding; based on the CLIP model and the ConvE model, it extracts the entity node feature embedding of the multimodal industrial product knowledge graph; The attention mechanism layer is used to use the time-sensitive attention mechanism based on the user behavior sequence characteristics to obtain the user's short-term interest characteristics; based on the entity node feature embedding, a multi-head attention mechanism is used to extract industrial product features; The intelligent recommendation and application evaluation layer is used to integrate the user's short-term interest characteristics with the industrial product characteristics to obtain the user's comprehensive preference characteristics; make recommendations based on the user's comprehensive preference characteristics, and score the user's comprehensive preference characteristics.

[0092] Embodiment 3 This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the multimodal industrial product recommendation method based on a hybrid attention mechanism as described in the first embodiment above are implemented.

[0093] Embodiment 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in a multimodal industrial product recommendation method based on a hybrid attention mechanism as described in the first embodiment above are implemented.

[0094] The steps or modules involved in the above embodiments 2 to 4 correspond to those in embodiment 1. For the specific implementation, please refer to the relevant description of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multimodal industrial product recommendation method based on a hybrid attention mechanism, characterized in that: include: Acquiring multimodal product data for preprocessing, wherein the multimodal product data includes text data and image data; Constructing a multimodal industrial product knowledge graph based on the preprocessed multimodal product data, wherein the multimodal industrial product knowledge graph includes a product information triple, a text triple, an image triple, and a user-product interaction quadruple; One-hot encoding is used to extract user behavior sequence features in user-product interaction quadruple. Based on CLIP model and ConvE model, entity node feature embedding of multimodal industrial product knowledge graph is extracted. Based on the user behavior sequence characteristics, the time-sensitive attention mechanism is used to obtain the user's short-term interest characteristics; based on the entity node feature embedding, the multi-head attention mechanism is used to extract the industrial product characteristics; Integrate user short-term interest characteristics with industrial product characteristics to obtain comprehensive user preference characteristics; Recommendations are made based on users' overall preference characteristics and scores are given to users' overall preference characteristics.

2. A multimodal industrial product recommendation method based on a hybrid attention mechanism as claimed in claim 1, characterized in that: The multimodal industrial product knowledge graph is constructed based on the preprocessed multimodal product data, specifically including: Obtain preprocessed text data and image data, extract product ID, relationship and attribute to construct product knowledge graph triples, extract product ID, description and text information to construct text triples, extract product ID, image and image information to construct image triples, extract user ID, product ID, rating and timestamp to construct user-product interaction quadruples.

3. The multimodal industrial product recommendation method based on a hybrid attention mechanism as claimed in claim 1, characterized in that: The method of extracting user behavior sequence features in the user-product interaction quadruple by using one-hot encoding specifically includes: Extract the user ID, the user-product interaction behavior and its corresponding timestamp from the user-product interaction quadruple, and obtain preliminary user features through one-hot encoding; Input the preliminary user features into the linear graph convolution module and the attention mechanism layer to obtain the user's product interaction sequence; The user behavior sequence features are obtained based on the user's product interaction sequence, interaction timestamp and product ID.

4. The multimodal industrial product recommendation method based on a hybrid attention mechanism as claimed in claim 1, characterized in that: The method extracts entity node feature embedding of the multimodal industrial product knowledge graph based on the CLIP model and the ConvE model; specifically includes: Extract text feature embedding of text triples and image feature embedding of image triples based on CLIP model; The ConvE model is used to extract user nodes, product nodes and attribute nodes from the user-product interaction quadruple and the product knowledge graph triple to obtain the initial entity features. Based on the initial entity features, the initial product entity feature embedding is obtained. The entity node feature embedding is obtained based on the text feature embedding, image feature embedding and initialized product entity feature embedding.

5. The multimodal industrial product recommendation method based on a hybrid attention mechanism as claimed in claim 3, characterized in that: The time-sensitive attention mechanism is adopted based on the user behavior sequence characteristics to obtain the user's short-term interest characteristics, specifically including: Extract the interaction time information from the user-product interaction quadruple and calculate the user's time interval characteristics; Construct the intermediate vector of the time-sensitive attention mechanism based on the time interval features and user behavior sequence features; Combine the preliminary user features with the intermediate vector of the time-sensitive attention mechanism and normalize them to obtain the normalized attention weights; The normalized attention weight and the user behavior sequence characteristics are weighted summed to obtain the user's short-term interest characteristics.

6. The multimodal industrial product recommendation method based on a hybrid attention mechanism as claimed in claim 1, characterized in that: The multi-head attention mechanism is adopted based on entity node feature embedding to extract industrial product features, specifically including: Calculate the attention score based on the entity node feature embedding; Based on the attention score, the adjacent embeddings of the head entity are summed to obtain the first-order structural information; The double interactive aggregation function is used to concatenate the structural information of all heads, and the head entity and its first-order structural information are aggregated. By stacking multiple attention layers, high-order structural information is captured to obtain industrial product features.

7. The multimodal industrial product recommendation method based on a hybrid attention mechanism as claimed in claim 1, characterized in that: The scoring of the user's comprehensive preference characteristics specifically includes: in, represents the comprehensive preference characteristics of users after fusion, The final representation of industrial product features after splicing different layers is shown. Indicates transpose.

8. A multimodal industrial product recommendation system based on a hybrid attention mechanism, characterized in that: include: A data processing layer, used to obtain multimodal product data for preprocessing, wherein the multimodal product data includes text data and image data; A knowledge graph layer, used to construct a multimodal industrial product knowledge graph based on the preprocessed multimodal product data, wherein the multimodal industrial product knowledge graph includes a product information triple, a text triple, an image triple, and a user-product interaction quadruple; The knowledge feature extraction layer is used to extract the user behavior sequence features in the user-product interaction quadruple using one-hot encoding; based on the CLIP model and the ConvE model, it extracts the entity node feature embedding of the multimodal industrial product knowledge graph; The attention mechanism layer is used to obtain the user's short-term interest features by using the time-sensitive attention mechanism based on the user's behavior sequence characteristics; Based on entity node feature embedding, a multi-head attention mechanism is used to extract industrial product features; Intelligent recommendation and application evaluation layer, used to integrate users’ short-term interest characteristics with industrial product characteristics to obtain users’ comprehensive preference characteristics; Recommendations are made based on users' overall preference characteristics and scores are given to users' overall preference characteristics.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a multimodal industrial product recommendation method based on a hybrid attention mechanism as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the multimodal industrial product recommendation method based on a hybrid attention mechanism as described in any one of claims 1-7 are implemented.

Images