Patent recommendation method and system based on multimodal heterogeneous graph

Abstract

The present invention relates to a patent recommendation method and system based on a multimodal heterogeneous graph, and belongs to the field of patent recommendation technology. The present invention fuses the first representation information and the second representation information corresponding to each attribute after analysis through a multi-layer perceptron model to generate a multi-modal heterogeneous graph fusion representation of the patent, and finally builds a patent recommendation model based on LightGCN, inputs the multi-modal heterogeneous graph fusion representation of the patent into the patent recommendation model for training, and obtains the trained patent recommendation model. The present invention performs multi-modal fusion recommendation on patent information by fusing the RoBerta model, the ResNet 50 model, the multi-layer perceptron model, and the LightGCN, thereby improving the recommendation accuracy of the patent recommendation model, and can more accurately recommend relevant patents to users, fully considering the characteristics of enterprise users and patent characteristics, which is conducive to improving the application and conversion of patents.

Description

Technical Field

[0001] The present invention relates to a patent recommendation method, and in particular to a patent recommendation method and system based on a multimodal heterogeneous graph. Background Art

[0002] Patent recommendation services are based on patent transfer histories, the company's technology areas, and the patent's technical features, providing companies with patent recommendations for new technological achievements. Patent similarity testing involves calculating the similarity between two or more patent documents to assess their degree of correlation. Patent similarity testing can help people better understand the correlations between patents, facilitate patent application, patent application, and patent infringement detection, and recommend appropriate patent achievements for practical production needs. In the field of patent analysis, the most common unimodal data type is text, so many patent similarity detection methods are based on patent text. While unimodal patent similarity detection methods have the advantages of simplicity and ease of implementation, they also have limitations, such as the inability to process multimodal data, the inability to consider semantic information in patent texts, and difficulty handling the complexity of patent texts. These limitations can also reduce the accuracy of patent recommendations, diminishing the user's patent search experience. Summary of the Invention

[0003] The present invention overcomes the deficiencies of the prior art and provides a patent recommendation method and system based on multimodal heterogeneous graphs.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is:

[0005] The first aspect of the present invention provides a patent recommendation method based on a multimodal heterogeneous graph, comprising the following steps:

[0006] Obtaining the first patent information, the second patent information, and the company information from the recommended patent library, introducing a ResNet50 model, and digitally representing the image information of the second patent using the ResNet50 model to generate first representation information;

[0007] Introducing the RoBerta model, and using the RoBerta model to digitally represent the text content of the first patent information and the enterprise information, generating second representation information corresponding to each attribute, and analyzing the second representation information corresponding to each attribute to generate analyzed second representation information corresponding to each attribute;

[0008] The first representation information and the second representation information corresponding to each analyzed attribute are fused through a multi-layer perceptron model to generate a patented multimodal heterogeneous graph fusion representation;

[0009] A patent recommendation model is constructed based on LightGCN, and the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training to obtain a trained patent recommendation model.

[0010] Furthermore, in this method, a ResNet50 model is introduced, and the second patent information is digitally represented by the ResNet50 model to generate first representation information, which specifically includes:

[0011] Obtain preprocessed image data information by filtering and denoising the second patent information, and input the preprocessed image data information into the ResNet50 model;

[0012] After the convolution and pooling operations of the first layer, it is determined whether the number of channels reaches the preset channel number threshold. If the number of channels does not reach the preset channel number threshold, the convolution and pooling operations are continued until the number of channels reaches the preset channel number threshold, and the image information after dimensionality reduction is obtained;

[0013] The reduced-dimensional image data is input into the residual module in the ResNet50 model, and after a preset number of residual operations, the digitally represented image data information is obtained, and the digitally represented image data information is output as the first representation information.

[0014] Furthermore, in this method, the RoBerta model is introduced, and the first patent information and enterprise information are digitally represented by the RoBerta model to generate second representation information corresponding to each attribute, specifically including:

[0015] Constructing a character vector, a sentence vector, and a position vector based on the first patent information and the enterprise information, introducing the RoBerta model, and constructing an embedding vector based on the character vector, sentence vector, and position vector;

[0016] The embedding vector is input into the RoBerta model, and after the embedding vector is processed by a preset number of layers of Transformer encoder units, the word vector representation corresponding to each attribute is finally output, and the word vector representation corresponding to each attribute is output as the second representation information corresponding to each attribute.

[0017] Furthermore, in this method, by analyzing the second representation information corresponding to each attribute, generating the analyzed second representation information corresponding to each attribute specifically includes:

[0018] Constructing a search tag according to the second representation information corresponding to each attribute, performing a search based on the search tag through big data acquisition to obtain text data information associated with the second representation information corresponding to each attribute, and introducing a multi-head attention mechanism;

[0019] Performing character segmentation on the text data information associated with the second representation information corresponding to each attribute to obtain a plurality of character information, and calculating correlation information between the character information and the second representation information corresponding to each attribute through the multi-head attention mechanism;

[0020] determining whether the association information is greater than a preset association threshold, and if so, constructing a directed edge description relationship, obtaining character information corresponding to the association information being greater than the preset association threshold, and constructing an associated word vector based on the character information corresponding to the association information being greater than the preset association threshold;

[0021] The associated word vector and the second representation information corresponding to each attribute are connected according to the directed edge description relationship, and the second representation information corresponding to each attribute is updated to generate the analyzed second representation information corresponding to each attribute.

[0022] Furthermore, in this method, the first representation information and the second representation information corresponding to each analyzed attribute are fused through a multi-layer perceptron model to generate a patented multimodal heterogeneous graph fusion representation, specifically including:

[0023] A multilayer perceptron model is introduced, and the first representation information and the second representation information corresponding to each analyzed attribute are input into the multilayer perceptron model, and an activation function RdLU and a Dropout mechanism are introduced between the first layer network and the second layer network of the multilayer perceptron model;

[0024] Obtaining a processed feature vector, applying node attribute and topology information guided contrast loss to the processed feature vector, and optimizing the feature vector generated by the second layer network so that nodes whose node attributes and topology information are less than a preset Mahalanobis distance value are close together in the feature space;

[0025] Nodes whose node attributes and topological information are not less than a preset Mahalanobis distance value are separated in the feature space, and the feature vectors generated by the second layer network are input into the third layer network. The cross entropy loss is calculated based on the feature vectors generated in the third layer network to determine whether the cross entropy loss meets the preset requirements;

[0026] When the cross entropy loss meets the preset requirements, the fused representation information is output, the fused representation information is used as the patented multimodal heterogeneous graph fusion representation, and the patented multimodal heterogeneous graph fusion representation is output.

[0027] Furthermore, in this method, a patent recommendation model is constructed based on LightGCN, and the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training to obtain a trained patent recommendation model, which specifically includes:

[0028] Build a patent recommendation model based on LightGCN, input the multimodal heterogeneous graph fusion representation of the patent into the patent recommendation model for training, and integrate the multimodal heterogeneous graph fusion representation of the patent into an embedding matrix in the initial embedding layer of the patent recommendation model;

[0029] Input the embedding matrix into the lightweight graph convolution layer of the patent recommendation model, sum the representation information in the embedding matrix of each layer, and obtain the final vector representation;

[0030] According to the final vector representation, an inner product operation is performed on the patent recommendation layer to obtain a prediction score. When the model parameters of the trained patent recommendation model meet the preset parameter requirements, the trained patent recommendation model is output.

[0031] A second aspect of the present invention provides a patent recommendation system based on a multimodal heterogeneous graph. The recommendation system includes a memory and a processor. The memory includes a patent recommendation method program based on a multimodal heterogeneous graph. When the patent recommendation method program based on a multimodal heterogeneous graph is implemented by the processor, the following steps are implemented:

[0032] Obtaining first patent information, second patent information, and enterprise information from the recommended patent library, introducing a ResNet50 model, and digitally representing the second patent information using the ResNet50 model to generate first representation information;

[0033] Introducing a RoBerta model, and digitally representing the first patent information and enterprise information using the RoBerta model to generate second representation information corresponding to each attribute, and analyzing the second representation information corresponding to each attribute to generate analyzed second representation information corresponding to each attribute;

[0034] The first representation information and the second representation information corresponding to each analyzed attribute are fused through a multi-layer perceptron model to generate a patented multimodal heterogeneous graph fusion representation;

[0035] A patent recommendation model is constructed based on LightGCN, and the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training to obtain a trained patent recommendation model.

[0036] The third aspect of the present invention provides a computer-readable storage medium, which includes a patent recommendation method program based on a multimodal heterogeneous graph. When the patent recommendation method program based on a multimodal heterogeneous graph is executed by a processor, it implements any step of the patent recommendation method based on a multimodal heterogeneous graph.

[0037] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0038] The present invention obtains the first patent information, the second patent information and the enterprise information in the recommended patent library, introduces the ResNet50 model, digitally represents the second patent information through the ResNet50 model, generates the first representation information, and then introduces the RoBerta model, and digitally represents the first patent information and the enterprise information through the RoBerta model to generate the second representation information corresponding to each attribute, and analyzes the second representation information corresponding to each attribute to generate the second representation information corresponding to the analyzed attribute, and then fuses the first representation information and the second representation information corresponding to each attribute after analysis through the multi-layer perceptron model to generate a multimodal heterogeneous graph fusion representation of the patent, and finally constructs a patent recommendation model based on LightGCN, and inputs the multimodal heterogeneous graph fusion representation of the patent into the patent recommendation model for training to obtain the trained patent recommendation model. The present invention integrates the RoBerta model, the multi-layer perceptron model and the LightGCN to fuse the claims, abstracts, abstract drawings, company names, business scope and other information in the patent information to form multimodal information, thereby improving the recommendation accuracy of the patent recommendation model, so that relevant patents can be recommended to users more accurately, fully considering the characteristics of corporate users and patents, which is conducive to improving the application and conversion of patents. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, without paying any creative work, they can also obtain drawings of other embodiments based on these drawings.

[0040] Figure 1 The overall method flow chart of the patent recommendation method based on multimodal heterogeneous graph is provided;

[0041] Figure 2 A flowchart of the first method of the patent recommendation method based on multimodal heterogeneous graph;

[0042] Figure 3 A flowchart of the second method of the patent recommendation method based on multimodal heterogeneous graph;

[0043] Figure 4 The system block diagram of the patent recommendation system based on multimodal heterogeneous graph. DETAILED DESCRIPTION

[0044] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0045] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0046] like Figure 1 The first aspect of the present invention provides a patent recommendation method based on a multimodal heterogeneous graph, comprising the following steps:

[0047] S102: Obtain first patent information, second patent information, and enterprise information from the recommended patent library, introduce a ResNet50 model, digitally represent the second patent information using the ResNet50 model, and generate first representation information;

[0048] S104: Introducing the RoBerta model, and digitally representing the first patent information and the enterprise information using the RoBerta model to generate second representation information corresponding to each attribute, and analyzing the second representation information corresponding to each attribute to generate analyzed second representation information corresponding to each attribute;

[0049] S106: Fusing the first representation information and the second representation information corresponding to each analyzed attribute through a multi-layer perceptron model to generate a patented multimodal heterogeneous graph fusion representation;

[0050] S108: Build a patent recommendation model based on LightGCN, input the multimodal heterogeneous graph fusion representation of the patent into the patent recommendation model for training, and obtain the trained patent recommendation model.

[0051] It should be noted that the present invention integrates the RoBerta model, the multi-layer perceptron model, and the LightGCN to fuse the claims, abstracts, abstract drawings, company names, business scopes, and other information in the patent information to form multimodal information, thereby improving the recommendation accuracy of the patent recommendation model, so that relevant patents can be recommended to users more accurately, fully considering the characteristics of corporate users and patents, and is conducive to improving the application and conversion of patents. Among them, the first patent information includes data such as claims, abstracts, keyword data, patent names, etc., and the company information includes information such as company names and business scopes.

[0052] Furthermore, in this method, a ResNet50 model is introduced to digitally represent the second patent information through the ResNet50 model to generate first representation information, specifically including:

[0053] By filtering and denoising the second patent information, preprocessed image data information is obtained, and the preprocessed image data information is input into the ResNet50 model;

[0054] After the convolution and pooling operations of the first layer, it is determined whether the number of channels reaches the preset channel number threshold. If the number of channels does not reach the preset channel number threshold, the convolution and pooling operations are continued until the number of channels reaches the preset channel number threshold, and the image information after dimensionality reduction is obtained;

[0055] The reduced-dimensional image data is input into the residual module in the ResNet50 model. After a preset number of residual operations, the digitally represented image data information is obtained, and the digitally represented image data information is output as the first representation information.

[0056] Among them, the second patent information is the specification drawings, abstract drawings, flow charts, etc.;

[0057] It should be noted that this method can convert the summary image into the first representation information of the model representation. Since ResNet50 has a total of 50 layers, the model input is 224×224 image data. After the first layer of convolution and pooling operations, the number of channels increases to 64, and the image dimension is reduced to 56×56. After entering the residual module, after four residual operations, the image feature map becomes (2048, 7, 7), thus forming the first representation information.

[0058] Furthermore, in this method, the RoBerta model is introduced, and the first patent information and enterprise information are digitally represented by the RoBerta model to generate second representation information corresponding to each attribute, specifically including:

[0059] Construct character vectors, sentence vectors, and position vectors based on the first patent information and enterprise information, introduce the RoBerta model, and construct embedding vectors based on the character vectors, sentence vectors, and position vectors;

[0060] The embedding vector is input into the RoBerta model. After the embedding vector is processed by a preset number of layers of Transformer encoder units, the word vector representation corresponding to each attribute is finally output, and the word vector representation corresponding to each attribute is output as the second representation information corresponding to each attribute.

[0061] It should be noted that the attributes include claims, keyword data, company name, company business scope, etc. Through this method, the second representation information output corresponding to each attribute can be obtained.

[0062] like Figure 2 As shown, further, in this method, by analyzing the second representation information corresponding to each attribute, generating the analyzed second representation information corresponding to each attribute, specifically includes:

[0063] S202: Constructing a search tag based on the second representation information corresponding to each attribute, performing a search based on the search tag through big data acquisition, obtaining text data information associated with the second representation information corresponding to each attribute, and introducing a multi-head attention mechanism;

[0064] S204: performing character segmentation on the text data information associated with the second representation information corresponding to each attribute, obtaining a plurality of character information, and calculating the correlation information between the character information and the second representation information corresponding to each attribute through a multi-head attention mechanism;

[0065] S206: Determine whether the relevance information is greater than a preset relevance threshold. If the relevance information is greater than the preset relevance threshold, construct a directed edge description relationship, obtain character information corresponding to the relevance information being greater than the preset relevance threshold, and construct a related word vector based on the character information corresponding to the relevance information being greater than the preset relevance threshold.

[0066] S208: Connect the associated word vector and the second representation information corresponding to each attribute according to the directed edge description relationship, and update the second representation information corresponding to each attribute to generate the analyzed second representation information corresponding to each attribute.

[0067] It should be noted that due to the polysemy of keywords in the data, the multi-head attention mechanism can calculate the correlation information between the character information and the second representation information corresponding to each attribute, so as to construct the associated word vector. For example, sodium carbonate has many nicknames, such as soda ash, caustic soda, etc. In the patent search, there may be data of this category but not retrieved. This method can construct the associated word vector, and connect the associated word vector and the second representation information corresponding to each attribute according to the directed edge description relationship. The second representation information corresponding to each attribute is updated to achieve data enhancement and improve the patent search accuracy.

[0068] like Figure 3 Furthermore, in this method, the first representation information and the second representation information corresponding to each analyzed attribute are fused through a multi-layer perceptron model to generate a patented multimodal heterogeneous graph fusion representation, specifically including:

[0069] S302: Introducing a multilayer perceptron model, inputting the first representation information and the second representation information corresponding to each analyzed attribute into the multilayer perceptron model, and introducing an activation function RdLU and a Dropout mechanism between the first layer network and the second layer network of the multilayer perceptron model;

[0070] S304: Obtain the processed feature vector, use node attributes and topology information to guide contrast loss on the processed feature vector, and optimize the feature vector generated by the second layer network so that nodes whose node attributes and topology information are less than a preset Mahalanobis distance value are close to each other in the feature space;

[0071] S306: Nodes whose node attributes and topology information are not less than a preset Mahalanobis distance value are separated in the feature space, and the feature vectors generated by the second layer network are input into the third layer network, and the cross entropy loss is calculated based on the feature vectors generated in the third layer network to determine whether the cross entropy loss meets the preset requirements;

[0072] S308: When the cross entropy loss meets the preset requirements, the fused representation information is output, the fused representation information is used as the multimodal heterogeneous graph fusion representation of the patent, and the multimodal heterogeneous graph fusion representation of the patent is output.

[0073] It should be noted that the multi-layer perceptron model is an MLP model. By introducing the activation function RdLU and Dropout mechanism between the first layer network and the second layer network of the multi-layer perceptron model, the nonlinear transformation capability of the model can be enhanced, and the overfitting of the parameters can be prevented, thereby improving the recommendation accuracy of the patent recommendation model.

[0074] Furthermore, in this method, a patent recommendation model is constructed based on LightGCN, and the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training to obtain the trained patent recommendation model, which specifically includes:

[0075] A patent recommendation model is built based on LightGCN. The multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training. In the initial embedding layer of the patent recommendation model, the multimodal heterogeneous graph fusion representation of the patent is integrated into an embedding matrix.

[0076] The embedding matrix is ​​input into the lightweight graph convolution layer of the patent recommendation model, and the representation information in the embedding matrix of each layer is summed to obtain the final vector representation;

[0077] According to the final vector representation, it is input into the patent recommendation layer for inner product operation to obtain the prediction score. When the model parameters of the trained patent recommendation model meet the preset parameter requirements, the trained patent recommendation model is output.

[0078] It should be noted that this method can obtain a trained patent recommendation model, so that users can input keyword data, patent information and other information into the trained patent recommendation model to further recommend data that better meets the requirements to users.

[0079] In addition, the method may further comprise the following steps:

[0080] Obtain the enterprise information corresponding to the patents to be transferred in the patent database, and search through channels based on the enterprise information corresponding to the recommended patents in the patent database to obtain the business information corresponding to the enterprise information corresponding to the recommended patents in the patent database;

[0081] Determining whether the business information corresponding to the enterprise information corresponding to the recommended patent in the patent database is the preset business information;

[0082] When the business information corresponding to the enterprise information in the recommended patent in the patent database is not the preset business information, the patent to be transferred is removed, and the patent to be transferred in the patent database is updated;

[0083] When the business situation information corresponding to the enterprise information in the recommended patent in the patent library is the preset business situation information, the corresponding patent to be transferred will be saved and the patent library will be updated regularly.

[0084] It should be noted that the preset operating status information is operating, wherein the operating status information includes descriptions of non-operating status, operating status, cancellation status, etc. This method can further improve the recommendation accuracy when searching for transferred patents.

[0085] In addition, the method may further comprise the following steps:

[0086] Obtain cancellation time information of related enterprises whose business conditions are about to be cancelled in the patents to be transferred through relevant channels, and construct a cancellation time sorting table, input the cancellation time information of related enterprises whose business conditions are about to be cancelled in the patents to be transferred into the cancellation time sorting table to sort the cancellation time in chronological order;

[0087] Obtaining a sorting time result by sorting, and presetting a logout time threshold range, and determining whether the logout time in the sorting time result is within the logout time threshold range;

[0088] When the cancellation time in the sorting time result is within the cancellation time threshold range, the recommendation priority of the corresponding patent to be transferred is increased; when the cancellation time in the sorting time result is not within the cancellation time threshold range, the recommendation priority of the corresponding patent to be transferred is decreased;

[0089] The latest recommended priority of the patent to be transferred is obtained, the transfer intention of the patent to be transferred is generated according to the latest recommended priority of the patent to be transferred, and the transfer intention of the patent to be transferred is displayed in a preset manner.

[0090] It should be noted that this method can further evaluate the situation of related companies to be transferred patents. When the cancellation time in the sorting time result is within the cancellation time threshold range, it means that the patent to be transferred is very likely to be transferred, liquidated, etc., thereby improving the priority of transfer recommendations, improving the recommendation accuracy of patents to be transferred, providing users with the highest transfer intentions and willingness, and improving the user's experience in searching for transferred patents.

[0091] like Figure 4 As shown, the second aspect of the present invention provides a patent recommendation system 4 based on a multimodal heterogeneous graph. The recommendation system 4 includes a memory 41 and a processor 42. The memory 41 includes a patent recommendation method program based on a multimodal heterogeneous graph. When the patent recommendation method program based on a multimodal heterogeneous graph is implemented by the processor 42, the following steps are implemented:

[0092] Obtaining the first patent information, the second patent information, and the enterprise information from the recommended patent library, introducing the ResNet50 model, digitally representing the second patent information through the ResNet50 model, and generating the first representation information;

[0093] Introducing the RoBerta model, and digitally representing the first patent information and the enterprise information using the RoBerta model to generate second representation information corresponding to each attribute, and analyzing the second representation information corresponding to each attribute to generate analyzed second representation information corresponding to each attribute;

[0094] The first representation information and the second representation information corresponding to each analyzed attribute are fused through a multi-layer perceptron model to generate a patented multimodal heterogeneous graph fusion representation;

[0095] A patent recommendation model is constructed based on LightGCN, and the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training to obtain the trained patent recommendation model.

[0096] Furthermore, in this system, the ResNet50 model is introduced to digitally represent the second patent information through the ResNet50 model to generate the first representation information, which specifically includes:

[0097] By filtering and denoising the second patent information, preprocessed image data information is obtained, and the preprocessed image data information is input into the ResNet50 model;

[0098] After the convolution and pooling operations of the first layer, it is determined whether the number of channels reaches the preset channel number threshold. If the number of channels does not reach the preset channel number threshold, the convolution and pooling operations are continued until the number of channels reaches the preset channel number threshold, and the image information after dimensionality reduction is obtained;

[0099] The reduced-dimensional image data is input into the residual module in the ResNet50 model. After a preset number of residual operations, the digitally represented image data information is obtained, and the digitally represented image data information is output as the first representation information.

[0100] Furthermore, in this system, the RoBerta model is introduced, and the first patent information and enterprise information are digitally represented by the RoBerta model to generate second representation information corresponding to each attribute, specifically including:

[0101] Construct character vectors, sentence vectors, and position vectors based on the first patent information and enterprise information, introduce the RoBerta model, and construct embedding vectors based on the character vectors, sentence vectors, and position vectors;

[0102] The embedding vector is input into the RoBerta model. After the embedding vector is processed by a preset number of layers of Transformer encoder units, the word vector representation corresponding to each attribute is finally output, and the word vector representation corresponding to each attribute is output as the second representation information corresponding to each attribute.

[0103] Furthermore, in this system, by analyzing the second representation information corresponding to each attribute, the analyzed second representation information corresponding to each attribute is generated, specifically including:

[0104] Construct a retrieval tag based on the second representation information corresponding to each attribute, perform a search based on the retrieval tag through big data acquisition, obtain text data information associated with the second representation information corresponding to each attribute, and introduce a multi-head attention mechanism;

[0105] Performing character segmentation on the text data information associated with the second representation information corresponding to each attribute to obtain a plurality of character information, and calculating the correlation information between the character information and the second representation information corresponding to each attribute through a multi-head attention mechanism;

[0106] Determine whether the relevance information is greater than a preset relevance threshold. If the relevance information is greater than the preset relevance threshold, construct a directed edge description relationship, obtain character information corresponding to the relevance information being greater than the preset relevance threshold, and construct a related word vector based on the character information corresponding to the relevance information being greater than the preset relevance threshold.

[0107] The associated word vector and the second representation information corresponding to each attribute are connected according to the directed edge description relationship, and the second representation information corresponding to each attribute is updated to generate the second representation information corresponding to each attribute after analysis.

[0108] Furthermore, in this system, the first representation information and the second representation information corresponding to each analyzed attribute are fused through a multi-layer perceptron model to generate a patented multimodal heterogeneous graph fusion representation, specifically including:

[0109] A multi-layer perceptron model is introduced, and the first representation information and the second representation information corresponding to each analyzed attribute are input into the multi-layer perceptron model, and an activation function RdLU and a Dropout mechanism are introduced between the first layer network and the second layer network of the multi-layer perceptron model;

[0110] Obtain the processed feature vector, use node attributes and topology information to guide contrast loss on the processed feature vector, and optimize the feature vector generated by the second layer network so that nodes whose node attributes and topology information are less than a preset Mahalanobis distance value are close together in the feature space;

[0111] The nodes whose node attributes and topological information are not less than the preset Mahalanobis distance value are separated in the feature space, and the feature vectors generated by the second layer network are input into the third layer network. The cross entropy loss is calculated based on the feature vectors generated in the third layer network to determine whether the cross entropy loss meets the preset requirements.

[0112] When the cross entropy loss meets the preset requirements, the fused representation information is output, the fused representation information is used as the multimodal heterogeneous graph fusion representation of the patent, and the multimodal heterogeneous graph fusion representation of the patent is output.

[0113] Furthermore, in this system, a patent recommendation model is constructed based on LightGCN, and the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training, and the trained patent recommendation model is obtained, which specifically includes:

[0114] A patent recommendation model is built based on LightGCN. The multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training. In the initial embedding layer of the patent recommendation model, the multimodal heterogeneous graph fusion representation of the patent is integrated into an embedding matrix.

[0115] The embedding matrix is ​​input into the lightweight graph convolution layer of the patent recommendation model, and the representation information in the embedding matrix of each layer is summed to obtain the final vector representation;

[0116] According to the final vector representation, it is input into the patent recommendation layer for inner product operation to obtain the prediction score. When the model parameters of the trained patent recommendation model meet the preset parameter requirements, the trained patent recommendation model is output.

[0117] The third aspect of the present invention provides a computer-readable storage medium, which includes a patent recommendation method program based on a multimodal heterogeneous graph. When the patent recommendation method program based on a multimodal heterogeneous graph is executed by a processor, it implements any step of the patent recommendation method based on a multimodal heterogeneous graph.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0119] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0120] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0121] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0122] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0123] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. Patent recommendation method based on multimodal heterogeneous graph, characterized in that: The following steps are involved: Obtaining the first patent information, the second patent information, and the enterprise information in the recommended patent library, introducing the ResNet50 model, digitally representing the second patent information through the ResNet50 model, and generating the first representation information; Introducing the RoBerta model, and digitally representing the first patent information and enterprise information through the RoBerta model to generate second representation information corresponding to each attribute, and generating second representation information corresponding to each attribute after analysis by analyzing the second representation information corresponding to each attribute; The first representation information and the second representation information corresponding to each attribute after analysis are fused through a multi-layer perceptron model to generate a patented multi-modal heterogeneous graph fusion representation; A patent recommendation model is constructed based on LightGCN, and the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training to obtain a trained patent recommendation model; A ResNet50 model is introduced, and the second patent information is digitally represented by the ResNet50 model to generate first representation information, specifically including: Obtain preprocessed image data information by filtering and denoising the second patent information, and input the preprocessed image data information into the ResNet50 model; After the convolution and pooling operations of the first layer, it is determined whether the number of channels reaches the preset channel number threshold. If the number of channels does not reach the preset channel number threshold, the convolution and pooling operations are continued until the number of channels reaches the preset channel number threshold, and the image information after dimensionality reduction is obtained; Input the reduced-dimensional image data into the residual module in the ResNet50 model, obtain digitally represented image data information after a preset number of residual operations, and output the digitally represented image data information as the first representation information; By analyzing the second representation information corresponding to each attribute, the analyzed second representation information corresponding to each attribute is generated, specifically including: Constructing a search tag according to the second representation information corresponding to each attribute, performing a search based on the search tag through big data acquisition, acquiring text data information associated with the second representation information corresponding to each attribute, and introducing a multi-head attention mechanism; Performing character segmentation on the text data information associated with the second representation information corresponding to each attribute to obtain a plurality of character information, and calculating the correlation information between the character information and the second representation information corresponding to each attribute through the multi-head attention mechanism; Determine whether the relevance information is greater than a preset relevance threshold, and when the relevance information is greater than the preset relevance threshold, construct a directed edge description relationship, obtain character information corresponding to the relevance information being greater than the preset relevance threshold, and construct a related word vector based on the character information corresponding to the relevance information being greater than the preset relevance threshold; The associated word vector and the second representation information corresponding to each attribute are connected according to the directed edge description relationship, and the second representation information corresponding to each attribute is updated to generate the analyzed second representation information corresponding to each attribute.

2. The patent recommendation method based on multimodal heterogeneous graph according to claim 1, characterized in that: The RoBerta model is introduced, and the first patent information and enterprise information are digitally represented by the RoBerta model to generate second representation information corresponding to each attribute, specifically including: Construct a character vector, a sentence vector, and a position vector according to the first patent information and the enterprise information, introduce the RoBerta model, and construct an embedding vector based on the character vector, the sentence vector, and the position vector; The embedding vector is input into the RoBerta model, and after the embedding vector is processed by a preset number of layers of Transformer encoder units, the word vector representation corresponding to each attribute is finally output, and the word vector representation corresponding to each attribute is output as the second representation information corresponding to each attribute.

3. The patent recommendation method based on multimodal heterogeneous graph according to claim 1, characterized in that: The first representation information and the second representation information corresponding to each attribute after analysis are fused through a multi-layer perceptron model to generate a patented multi-modal heterogeneous graph fusion representation, which specifically includes: A multi-layer perceptron model is introduced, and the first representation information and the second representation information corresponding to each attribute after analysis are input into the multi-layer perceptron model, and an activation function RdLU and a Dropout mechanism are introduced between the first layer network and the second layer network of the multi-layer perceptron model; Obtaining a processed feature vector, using node attributes and topological information to guide contrast loss on the processed feature vector, and optimizing the feature vector generated by the second layer network so that nodes whose node attributes and topological information are less than a preset Mahalanobis distance value are close in the feature space; Make nodes whose node attributes and topological information are not less than a preset Mahalanobis distance value far away in the feature space, input the feature vector generated by the second layer network into the third layer network, calculate the cross entropy loss according to the feature vector generated in the third layer network, and judge whether the cross entropy loss meets the preset requirements; When the cross entropy loss meets the preset requirements, the fused representation information is output, the fused representation information is used as the multimodal heterogeneous graph fusion representation of the patent, and the multimodal heterogeneous graph fusion representation of the patent is output.

4. The patent recommendation method based on multimodal heterogeneous graph according to claim 1, characterized in that: A patent recommendation model is constructed based on LightGCN, and the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training to obtain the trained patent recommendation model, which specifically includes: A patent recommendation model is constructed based on LightGCN, the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training, and in the initial embedding layer of the patent recommendation model, the multimodal heterogeneous graph fusion representation of the patent is integrated into an embedding matrix; Input the embedding matrix into the lightweight graph convolution layer of the patent recommendation model, sum the representation information in the embedding matrix of each layer, and obtain the final vector representation; According to the final vector representation, an inner product operation is performed on the patent recommendation layer to obtain a prediction score. When the model parameters of the trained patent recommendation model meet the preset parameter requirements, the trained patent recommendation model is output.

5. A patent recommendation system based on multimodal heterogeneous graph, characterized in that: The recommendation system includes a memory and a processor, wherein the memory includes a patent recommendation method program based on a multimodal heterogeneous graph, and when the patent recommendation method program based on a multimodal heterogeneous graph is implemented by the processor, the following steps are implemented: Obtaining the first patent information, the second patent information, and the enterprise information in the recommended patent library, introducing the ResNet50 model, digitally representing the second patent information through the ResNet50 model, and generating the first representation information; Introducing the RoBerta model, and digitally representing the first patent information and enterprise information through the RoBerta model to generate second representation information corresponding to each attribute, and generating second representation information corresponding to each attribute after analysis by analyzing the second representation information corresponding to each attribute; The first representation information and the second representation information corresponding to each attribute after analysis are fused through a multi-layer perceptron model to generate a patented multi-modal heterogeneous graph fusion representation; A patent recommendation model is constructed based on LightGCN, and the multimodal heterogeneous graph fusion representation of the patent is input into the patent recommendation model for training to obtain a trained patent recommendation model; A ResNet50 model is introduced, and the second patent information is digitally represented by the ResNet50 model to generate first representation information, specifically including: Obtain preprocessed image data information by filtering and denoising the second patent information, and input the preprocessed image data information into the ResNet50 model; After the convolution and pooling operations of the first layer, it is determined whether the number of channels reaches the preset channel number threshold. If the number of channels does not reach the preset channel number threshold, the convolution and pooling operations are continued until the number of channels reaches the preset channel number threshold, and the image information after dimensionality reduction is obtained; Input the reduced-dimensional image data into the residual module in the ResNet50 model, obtain digitally represented image data information after a preset number of residual operations, and output the digitally represented image data information as the first representation information; By analyzing the second representation information corresponding to each attribute, the analyzed second representation information corresponding to each attribute is generated, specifically including: Constructing a search tag according to the second representation information corresponding to each attribute, performing a search based on the search tag through big data acquisition, acquiring text data information associated with the second representation information corresponding to each attribute, and introducing a multi-head attention mechanism; Performing character segmentation on the text data information associated with the second representation information corresponding to each attribute to obtain a plurality of character information, and calculating the correlation information between the character information and the second representation information corresponding to each attribute through the multi-head attention mechanism; Determine whether the relevance information is greater than a preset relevance threshold, and when the relevance information is greater than the preset relevance threshold, construct a directed edge description relationship, obtain character information corresponding to the relevance information being greater than the preset relevance threshold, and construct a related word vector based on the character information corresponding to the relevance information being greater than the preset relevance threshold; The associated word vector and the second representation information corresponding to each attribute are connected according to the directed edge description relationship, and the second representation information corresponding to each attribute is updated to generate the analyzed second representation information corresponding to each attribute.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a patent recommendation method program based on a multimodal heterogeneous graph. When the patent recommendation method program based on a multimodal heterogeneous graph is executed by a processor, the steps of the patent recommendation method based on a multimodal heterogeneous graph as described in any one of claims 1 to 4 are implemented.

Images