Project subject word extraction method, device, equipment and medium

Abstract

The application belongs to the technical field of data processing, and discloses a project subject word extraction method, device, equipment and medium. The method comprises the following steps: obtaining a text modal representation of a project, and obtaining an image modal representation of the project; performing modal alignment on the text modal representation and the image modal representation to obtain an alignment vector; determining an initial weight of the alignment vector, and taking a logarithmic operation result of the initial weight as an attention weight of the alignment vector; inputting the alignment vector and the attention weight into a deep learning model of a preset network model to obtain deep semantic information; and predicting the deep semantic information through a regression model of the preset network model to obtain a project subject word of the project. In the above manner, the accuracy of project subject word extraction can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and medium for extracting project keywords. Background Technology

[0002] Existing methods for extracting project keywords mainly use text to extract keywords, but this method cannot fully obtain project information, which affects the accuracy of keyword extraction. Summary of the Invention

[0003] The main objective of this invention is to provide a method, apparatus, device, and medium for extracting project keywords, aiming to solve the technical problem in the prior art where the inability to fully obtain project information affects the accuracy of keyword extraction.

[0004] To achieve the above objectives, the present invention provides a method for extracting project keywords, the method comprising the following steps:

[0005] Obtain the text modal representation of the project, and obtain the image modal representation of the project;

[0006] The text modal representation and the image modal representation are modally aligned to obtain an alignment vector;

[0007] Determine the initial weights of the alignment vector, and use the logarithmic result of the initial weights as the attention weights of the alignment vector;

[0008] The alignment vector and the attention weights are input into a deep learning model of a preset network model to obtain deep semantic information;

[0009] The project's keyword is obtained by predicting the deep semantic information using the regression model of the preset network model.

[0010] Optionally, obtaining the text modal representation of the project includes:

[0011] Obtain labeled text data, wherein the labeled text data is labeled with text tags;

[0012] The labeled text data is input into a preset recurrent neural network to obtain text prediction values;

[0013] Based on the text labels and the text prediction values, determine the text loss value;

[0014] The preset recurrent neural network is updated based on the text loss value to obtain the trained preset recurrent neural network.

[0015] The text data of the project is input into the trained preset recurrent neural network to obtain the text modal representation of the project.

[0016] Optionally, obtaining the image modal representation of the project includes:

[0017] Obtain labeled image data, wherein the labeled image data is labeled with image tags;

[0018] The image mode matrix of the labeled image data is determined by using convolution and pooling operations of a convolutional neural network.

[0019] The image modality matrix is ​​input into the fully connected layer or global mean pooling layer of the convolutional neural network, and the image prediction value is output.

[0020] The image loss value is determined based on the image label and the image prediction value;

[0021] The convolutional neural network is updated based on the image loss value to obtain a trained convolutional neural network;

[0022] The image data of the project is input into the trained convolutional neural network to obtain the image modal representation of the project.

[0023] Optionally, the convolutional neural network is a CapsNet neural network, which is composed of capsule neurons. Adjacent capsule layers in the CapsNet neural network are connected via a dynamic routing algorithm. The CapsNet neural network uses a dynamic routing algorithm instead of pooling layers in the convolutional neural network.

[0024] The step of inputting the image data of the project into the trained convolutional neural network to obtain the image modal representation of the project includes:

[0025] The image data is represented by a vector representation of instance orientation and spatial information by the capsule layer of the CapsNet neural network, and the image modality representation of the item is determined based on the vector representation.

[0026] Optionally, the step of modally aligning the text modal representation and the image modal representation to obtain an alignment vector includes:

[0027] Acquire text data and image data, and construct a semantic alignment dataset based on the text data and the image data, wherein the data in the semantic alignment dataset are labeled with relational tags;

[0028] A deep learning model is trained using the semantic alignment dataset to obtain a well-trained deep learning model.

[0029] Based on the trained deep learning model and Markov random field, the text modal representation and the image modal representation are modally aligned to obtain an alignment vector.

[0030] Optionally, determining the initial weights of the alignment vector and using the logarithmic result of the initial weights as the attention weights of the alignment vector includes:

[0031] Determine the product of any vector in the alignment vector with other vectors, and determine the initial weight of each vector in the alignment vector based on the product;

[0032] The logarithmic result of the initial weights is determined, and the attention weight of each vector in the alignment vector is determined based on the logarithmic result.

[0033] Optionally, the step of predicting the deep semantic information using the regression model of the preset network model to obtain the project's topic keywords includes:

[0034] The deep semantic information is predicted using the regression model of the preset network model, the prediction probability between regional target words and target words of the deep semantic information is determined, and the probability matrix is ​​determined based on the prediction probability.

[0035] The target word with the highest predicted probability in the probability matrix is ​​taken as the project keyword.

[0036] Furthermore, to achieve the above objectives, the present invention also proposes a project keyword extraction device, which includes:

[0037] The acquisition module is used to acquire the text modal representation of the project and the image modal representation of the project;

[0038] The determining module is used to perform modal alignment between the text modal representation and the image modal representation to obtain an alignment vector;

[0039] The determining module is further configured to determine the initial weights of the alignment vector and use the logarithmic result of the initial weights as the attention weights of the alignment vector;

[0040] The determining module is further configured to input the alignment vector and the attention weight into the deep learning model of the preset network model to obtain deep semantic information;

[0041] The prediction module is used to predict the deep semantic information through the regression model of the preset network model to obtain the project topic words.

[0042] Furthermore, to achieve the above objectives, the present invention also proposes a project keyword extraction device, which includes: a memory, a processor, and a project keyword extraction program stored in the memory and executable on the processor. The project keyword extraction program is configured to implement the steps of the project keyword extraction method described above.

[0043] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a project keyword extraction program, which, when executed by a processor, implements the steps of the project keyword extraction method described above.

[0044] The present invention proposes a method, apparatus, device, and medium for extracting project keywords. This involves acquiring a textual modal representation and an image modal representation of the project; aligning the textual and image modal representations to obtain an alignment vector; determining the initial weights of the alignment vector and using the logarithmic result of these initial weights as the attention weights; inputting the alignment vector and the attention weights into a deep learning model of a preset network model to obtain deep semantic information; and predicting the deep semantic information using a regression model of the preset network model to obtain the project keywords. By combining textual and image data to extract project keywords, a more comprehensive understanding of project information can be obtained. Furthermore, the combination of deep learning and regression models in keyword extraction allows for a more accurate understanding of semantics, thereby effectively improving the accuracy of project keyword extraction. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of the project keyword extraction device in the hardware operating environment involved in the embodiments of the present invention;

[0046] Figure 2 This is a flowchart illustrating the first embodiment of the keyword extraction method of the present invention.

[0047] Figure 3 This is a flowchart illustrating the process of obtaining project keywords in the first embodiment of the project keyword extraction method of the present invention.

[0048] Figure 4 This is a flowchart illustrating the second embodiment of the keyword extraction method for this invention.

[0049] Figure 5 This is a structural block diagram of the first embodiment of the subject term extraction device of the present invention.

[0050] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0051] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0052] Reference Figure 1 , Figure 1 This is a schematic diagram of the hardware operating environment for extracting project keywords in an embodiment of the present invention.

[0053] like Figure 1 As shown, the subject extraction device for this project may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0054] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the project keyword extraction device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0055] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a project keyword extraction program.

[0056] exist Figure 1In the project keyword extraction device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the project keyword extraction device of the present invention can be set in the project keyword extraction device, and the project keyword extraction device calls the project keyword extraction program stored in the memory 1005 through the processor 1001 and executes the project keyword extraction method provided in the embodiment of the present invention.

[0057] Based on the above hardware structure, an embodiment of the topic word extraction method of the present invention is proposed.

[0058] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of a project keyword extraction method according to the present invention.

[0059] In this embodiment, the project keyword extraction method includes the following steps:

[0060] Step S10: Obtain the text modal representation of the project and the image modal representation of the project.

[0061] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a mobile phone, tablet computer, or personal computer, or an electronic device or project keyword extraction device capable of performing the above functions. The following description uses the project keyword extraction device as an example to illustrate this embodiment and the subsequent embodiments.

[0062] It should be noted that text modality representation refers to mapping a text sequence to a semantic space; image modality representation refers to mapping an image sequence to a semantic space, text modality representation, and obtaining the image modality representation of the item.

[0063] In the specific implementation, it is necessary to first obtain the text data and image data of the project in order to determine the text modal representation of the project based on the text data and the image modal representation of the project based on the image data. For example, the shape of the text modal representation is (3, 512, 512) and the shape of the image modal representation is (3, 512, 512).

[0064] In one embodiment, obtaining the text modal representation of the project includes:

[0065] Obtain labeled text data, wherein the labeled text data is labeled with text tags;

[0066] The labeled text data is input into a preset recurrent neural network to obtain text prediction values;

[0067] Based on the text labels and the text prediction values, determine the text loss value;

[0068] The preset recurrent neural network is updated based on the text loss value to obtain the trained preset recurrent neural network.

[0069] The text data of the project is input into the trained preset recurrent neural network to obtain the text modal representation of the project.

[0070] It should be noted that text modality representation refers to mapping text sequences (i.e., the text data of items) to a semantic space; the presupposed recurrent neural network is a recurrent neural network based on the self-attention mechanism; due to the variable length of the input sequence of the recurrent neural network and the fact that the current output is related to the previous input, the recurrent neural network has become the mainstream model for nonlinear mapping in sentence modality processing.

[0071] In the specific implementation, after setting the parameters such as Optimizer, learning_rate, batch_size, and epoch for training the preset recurrent neural network model, training is carried out. During the training process, the preset recurrent neural network needs to be continuously updated based on the text loss value. When the text loss value is less than or equal to the set threshold, the training of the preset recurrent neural network can be stopped, thus obtaining the trained preset recurrent neural network.

[0072] In this embodiment, by using a preset recurrent neural network to obtain the text modal representation of the project, it is possible not only to obtain the vector representation of the current text based on the context, but also to better map the text data of the project into the semantic space.

[0073] In one embodiment, obtaining the image modal representation of the project includes:

[0074] Obtain labeled image data, wherein the labeled image data is labeled with image tags;

[0075] The image mode matrix of the labeled image data is determined by using convolution and pooling operations of a convolutional neural network.

[0076] The image modality matrix is ​​input into the fully connected layer or global mean pooling layer of the convolutional neural network, and the image prediction value is output.

[0077] The image loss value is determined based on the image label and the image prediction value;

[0078] The convolutional neural network is updated based on the image loss value to obtain a trained convolutional neural network;

[0079] The image data of the project is input into the trained convolutional neural network to obtain the image modal representation of the project.

[0080] It should be noted that convolutional neural networks (CNNs) are a deep learning method specifically designed for images, developed on the basis of multi-layer neural networks. The convolution and pooling operations of CNNs can be understood as the process of generating image modality matrices. The output of the fully connected layers or global mean pooling layers of CNNs can be vector representations of modalities. Classic CNNs include LeNet-5, AlexNet, VGG, GoogLeNet, ResNet, and CapsNet.

[0081] In this embodiment, by employing a convolutional neural network to obtain the image modal representation of the project, the loss of effective information in the image can be avoided.

[0082] In one embodiment, the convolutional neural network is a CapsNet neural network, which is composed of capsule neurons. Adjacent capsule layers in the CapsNet neural network are connected via a dynamic routing algorithm. The CapsNet neural network uses a dynamic routing algorithm instead of pooling layers in the convolutional neural network.

[0083] The step of inputting the image data of the project into the trained convolutional neural network to obtain the image modal representation of the project includes:

[0084] The image data is represented by a vector representation of instance orientation and spatial information by the capsule layer of the CapsNet neural network, and the image modality representation of the item is determined based on the vector representation.

[0085] It's important to note that CapsNet is an extension of convolutional neural networks. Its basic building block is the capsule; a capsule is a group of neurons whose inputs and outputs are vectors. Each element in the vector represents a parameter of a feature of an entity in the image, and adjacent capsule layers are connected through a dynamic routing algorithm to select parameters. Therefore, the CapsNet neural network can generate vector representations containing the orientation and spatial information of instances in the image at each capsule layer. Furthermore, it uses a dynamic routing algorithm instead of pooling layers in convolutional neural networks, avoiding the loss of useful information. By training the CapsNet neural network with labeled image data, it can obtain image modal representations of items.

[0086] In this embodiment, by using the CapsNet neural network to obtain the image modal representation of the project, not only can the loss of effective information in the image be avoided, but parameter selection can also be achieved through the dynamic routing algorithm in the CapsNet neural network.

[0087] Step S20: Perform modal alignment between the text modal representation and the image modal representation to obtain an alignment vector.

[0088] It should be noted that modal alignment refers to identifying the relationship between elements from two or more different modalities; modal alignment is used to identify the relationship between the labeling statements of an item and the corresponding words or phrases in each region of the item's image; semantic alignment can be used to align the text modal representation and the image modal representation.

[0089] In one embodiment, the step of modally aligning the text modal representation and the image modal representation to obtain an alignment vector includes:

[0090] Acquire text data and image data, and construct a semantic alignment dataset based on the text data and the image data, wherein the data in the semantic alignment dataset are labeled with relational tags;

[0091] A deep learning model is trained using the semantic alignment dataset to obtain a well-trained deep learning model;

[0092] Based on the trained deep learning model and Markov random field, the text modal representation and the image modal representation are modally aligned to obtain an alignment vector.

[0093] It should be noted that semantic alignment is an alignment method that directly endows a model with alignment capabilities. The main way to implement semantic alignment is to process data with relational labels and generate a semantic alignment dataset. Then, a deep learning model is used to learn the semantic alignment information in the semantic alignment dataset. The deep learning model is trained with an objective function containing semantic information. Then, through the trained deep learning model and the newly constructed chain-structured Markov random field, the energy function is dynamically minimized to find the best semantically aligned image and sentence or word pair, thereby obtaining the alignment vector.

[0094] Step S30: Determine the initial weights of the alignment vector, and use the logarithmic result of the initial weights as the attention weights of the alignment vector.

[0095] It should be noted that, firstly, each individual vector in the modally aligned vector is multiplied with the other vectors in turn, and then the product result is used as the initial weight of each vector in the aligned vector. Finally, the logarithmic result of the initial weights is used as the attention weight of the aligned vector.

[0096] Step S40: Input the alignment vector and the attention weight into the deep learning model of the preset network model to obtain deep semantic information.

[0097] In the specific implementation, taking the LSTM model as an example of a deep learning model, the alignment vector and attention weights are input into the LSTM model for feature extraction to obtain deep semantic information. The deep semantic information reflects the inherent relationship between the text data and image data of the project.

[0098] Step S50: Predict the deep semantic information using the regression model of the preset network model to obtain the project topic words.

[0099] In specific implementations, such as Figure 3 As shown, the steps for obtaining project topic words include multimodal representation, modality alignment, and topic extraction. Multimodal representation includes obtaining text modality representation and image modality representation of the project. Modality alignment is performed by aligning the text modality representation and image modality representation. Topic extraction uses an LSTM model to learn the deep semantics of text and images and then predicts the target words.

[0100] In one embodiment, the step of predicting the deep semantic information using a regression model of the preset network model to obtain the project's topic keywords includes:

[0101] The deep semantic information is predicted using the regression model of the preset network model, the prediction probability between regional target words and target words of the deep semantic information is determined, and the probability matrix is ​​determined based on the prediction probability.

[0102] The target word with the highest predicted probability in the probability matrix is ​​taken as the project keyword.

[0103] It should be noted that regression models include logistic regression, decision tree regression, and ridge regression.

[0104] In the specific implementation, taking the ridge regression algorithm as an example, the ridge regression algorithm is used to predict deep semantic information, obtain the predicted probabilities between the regional target words and all target words in the deep semantic information, and obtain a probability matrix. The target word with the highest preset probability is taken from the probability matrix as the topic word of the project.

[0105] In this implementation, the LSTM model is combined with ridge regression for topic extraction, which can greatly improve the generalization ability and accuracy of the topic extraction model. Moreover, the model only needs to be trained once and can be used directly thereafter, making it highly versatile.

[0106] This embodiment obtains the textual modal representation and the image modal representation of the project; it then aligns the textual and image modal representations to obtain an alignment vector; it determines the initial weights of the alignment vector and uses the logarithmic result of the initial weights as the attention weights of the alignment vector; it inputs the alignment vector and the attention weights into a deep learning model of a preset network model to obtain deep semantic information; and it uses a regression model of the preset network model to predict the deep semantic information to obtain the project's topic words. By combining textual and image data to extract topic words, this method provides a more comprehensive understanding of project information. Furthermore, the use of a combination of a deep learning LSTM model and ridge regression in the regression model for topic extraction allows for a more accurate understanding of the semantics, thereby effectively improving the accuracy of topic word extraction.

[0107] refer to Figure 4 , Figure 4 This is a flowchart illustrating a second embodiment of a project keyword extraction method according to the present invention.

[0108] Based on the first embodiment described above, the topic word extraction method of this embodiment, which determines the initial weights of the alignment vector and uses the logarithmic result of the initial weights as the attention weights of the alignment vector, includes:

[0109] Step S301: Determine the product of any vector in the alignment vector with other vectors, and determine the initial weight of each vector in the alignment vector based on the product.

[0110] It should be noted that, firstly, each vector in the alignment vector after modal alignment is multiplied with the other vectors in turn, and then the product result is used as the initial weight of each vector in the alignment vector.

[0111] Step S302: Determine the logarithmic result of the initial weights, and determine the attention weights of each vector in the alignment vector based on the logarithmic result.

[0112] In practice, the logarithm of the initial weights to the base 10 is the result of the logarithmic operation of the initial weights.

[0113] This embodiment determines the product of any vector in the alignment vector with other vectors, and determines the initial weight of each vector in the alignment vector based on the product; it then determines the logarithmic result of the initial weights, and determines the attention weight of each vector in the alignment vector based on the logarithmic result. By doing so, the influence of negative weights on the final prediction of the sentence can be eliminated, thereby effectively improving the prediction accuracy of project topic words.

[0114] Furthermore, this embodiment of the invention also proposes a storage medium storing a project keyword extraction program, which, when executed by a processor, implements the steps of the project keyword extraction method described above.

[0115] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the subject term extraction device of the present invention.

[0116] like Figure 5 As shown, the project keyword extraction device proposed in this embodiment of the invention includes:

[0117] The acquisition module 10 is used to acquire the text modal representation of the project and the image modal representation of the project.

[0118] The determining module 20 is used to perform modal alignment between the text modal representation and the image modal representation to obtain an alignment vector.

[0119] The determining module 20 is further configured to determine the initial weights of the alignment vector and use the logarithmic result of the initial weights as the attention weights of the alignment vector.

[0120] The determining module 20 is further configured to input the alignment vector and the attention weight into the deep learning model of the preset network model to obtain deep semantic information.

[0121] The prediction module 30 is used to predict the deep semantic information through the regression model of the preset network model to obtain the project topic words.

[0122] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0123] This embodiment obtains the textual modal representation and the image modal representation of the project; it then aligns the textual and image modal representations to obtain an alignment vector; it determines the initial weights of the alignment vector and uses the logarithmic result of the initial weights as the attention weights of the alignment vector; it inputs the alignment vector and the attention weights into a deep learning model of a preset network model to obtain deep semantic information; and it uses a regression model of the preset network model to predict the deep semantic information to obtain the project's topic words. By combining textual and image data to extract topic words, this method provides a more comprehensive understanding of project information. Furthermore, the use of a combination of a deep learning LSTM model and ridge regression in the regression model for topic extraction allows for a more accurate understanding of the semantics, thereby effectively improving the accuracy of topic word extraction.

[0124] In one embodiment, the acquisition module 10 is further configured to:

[0125] Obtain labeled text data, wherein the labeled text data is labeled with text tags;

[0126] The labeled text data is input into a preset recurrent neural network to obtain text prediction values;

[0127] Based on the text labels and the text prediction values, determine the text loss value;

[0128] The preset recurrent neural network is updated based on the text loss value to obtain the trained preset recurrent neural network.

[0129] The text data of the project is input into the trained preset recurrent neural network to obtain the text modal representation of the project.

[0130] In one embodiment, the acquisition module 10 is further configured to:

[0131] Obtain labeled image data, wherein the labeled image data is labeled with image tags;

[0132] The image mode matrix of the labeled image data is determined by using convolution and pooling operations of a convolutional neural network.

[0133] The image modality matrix is ​​input into the fully connected layer or global mean pooling layer of the convolutional neural network, and the image prediction value is output.

[0134] The image loss value is determined based on the image label and the image prediction value;

[0135] The convolutional neural network is updated based on the image loss value to obtain a trained convolutional neural network;

[0136] The image data of the project is input into the trained convolutional neural network to obtain the image modal representation of the project.

[0137] In one embodiment, the convolutional neural network is a CapsNet neural network, which is composed of capsule neurons. Adjacent capsule layers in the CapsNet neural network are connected via a dynamic routing algorithm. The CapsNet neural network uses a dynamic routing algorithm instead of pooling layers in the convolutional neural network.

[0138] The acquisition module 10 is further configured to:

[0139] The image data is represented by a vector representation of instance orientation and spatial information by the capsule layer of the CapsNet neural network, and the image modality representation of the item is determined based on the vector representation.

[0140] In one embodiment, the determining module 20 is further configured to:

[0141] Acquire text data and image data, and construct a semantic alignment dataset based on the text data and the image data, wherein the data in the semantic alignment dataset are labeled with relational tags;

[0142] A deep learning model is trained using the semantic alignment dataset to obtain a well-trained deep learning model;

[0143] Based on the trained deep learning model and Markov random field, the text modal representation and the image modal representation are modally aligned to obtain an alignment vector.

[0144] In one embodiment, the determining module 20 is further configured to:

[0145] Determine the product of any vector in the alignment vector with other vectors, and determine the initial weight of each vector in the alignment vector based on the product;

[0146] The logarithmic result of the initial weights is determined, and the attention weight of each vector in the alignment vector is determined based on the logarithmic result.

[0147] In one embodiment, the prediction module 30 is further configured to:

[0148] The deep semantic information is predicted using the regression model of the preset network model, the prediction probability between regional target words and target words of the deep semantic information is determined, and the probability matrix is ​​determined based on the prediction probability.

[0149] The target word with the highest predicted probability in the probability matrix is ​​taken as the project keyword.

[0150] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0151] In addition, for technical details not described in detail in this embodiment, please refer to the project keyword extraction method provided in any embodiment of the present invention, which will not be repeated here.

[0152] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0153] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0154] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0155] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for extracting project keywords, characterized in that, The project keyword extraction method includes: Obtain the text modal representation of the project, and obtain the image modal representation of the project; The text modal representation and the image modal representation are modally aligned to obtain an alignment vector; Determine the initial weights of the alignment vector, and use the logarithmic result of the initial weights as the attention weights of the alignment vector; The alignment vector and the attention weights are input into a deep learning model of a preset network model to obtain deep semantic information; The project's keyword is obtained by predicting the deep semantic information using the regression model of the preset network model.

2. The method as described in claim 1, characterized in that, The obtained text modal representation of the project includes: Obtain labeled text data, wherein the labeled text data is labeled with text tags; The labeled text data is input into a preset recurrent neural network to obtain text prediction values; Based on the text labels and the text prediction values, determine the text loss value; The preset recurrent neural network is updated based on the text loss value to obtain the trained preset recurrent neural network. The text data of the project is input into the trained preset recurrent neural network to obtain the text modal representation of the project.

3. The method as described in claim 1, characterized in that, The acquisition of the image modal representation of the project includes: Obtain labeled image data, wherein the labeled image data is labeled with image tags; The image mode matrix of the labeled image data is determined by using convolution and pooling operations of a convolutional neural network. The image modality matrix is ​​input into the fully connected layer or global mean pooling layer of the convolutional neural network, and the image prediction value is output. The image loss value is determined based on the image label and the image prediction value; The convolutional neural network is updated based on the image loss value to obtain a trained convolutional neural network; The image data of the project is input into the trained convolutional neural network to obtain the image modal representation of the project.

4. The method as described in claim 3, characterized in that, The convolutional neural network is a CapsNet neural network, which is composed of capsule neurons. Adjacent capsule layers in the CapsNet neural network are connected via a dynamic routing algorithm. The CapsNet neural network uses a dynamic routing algorithm instead of the pooling layers in the convolutional neural network. The step of inputting the image data of the project into the trained convolutional neural network to obtain the image modal representation of the project includes: The image data is represented by a vector representation of instance orientation and spatial information by the capsule layer of the CapsNet neural network, and the image modality representation of the item is determined based on the vector representation.

5. The method as described in claim 1, characterized in that, The step of modal aligning the text modal representation and the image modal representation to obtain an alignment vector includes: Acquire text data and image data, and construct a semantic alignment dataset based on the text data and the image data, wherein the data in the semantic alignment dataset are labeled with relational tags; A deep learning model is trained using the semantic alignment dataset to obtain a well-trained deep learning model. Based on the trained deep learning model and Markov random field, the text modal representation and the image modal representation are modally aligned to obtain an alignment vector.

6. The method as described in claim 1, characterized in that, The step of determining the initial weights of the alignment vector and using the logarithmic result of the initial weights as the attention weights of the alignment vector includes: Determine the product of any vector in the alignment vector with other vectors, and determine the initial weight of each vector in the alignment vector based on the product; The logarithmic result of the initial weights is determined, and the attention weight of each vector in the alignment vector is determined based on the logarithmic result.

7. The method as described in claim 1, characterized in that, The project topic keywords are obtained by predicting the deep semantic information using a regression model based on the preset network model, including: The deep semantic information is predicted using the regression model of the preset network model, the prediction probability between regional target words and target words of the deep semantic information is determined, and the probability matrix is ​​determined based on the prediction probability. The target word with the highest predicted probability in the probability matrix is ​​taken as the project keyword.

8. A project keyword extraction device, characterized in that, The project keyword extraction device includes: The acquisition module is used to acquire the text modal representation of the project and the image modal representation of the project; The determining module is used to perform modal alignment between the text modal representation and the image modal representation to obtain an alignment vector; The determining module is further configured to determine the initial weights of the alignment vector and use the logarithmic result of the initial weights as the attention weights of the alignment vector; The determining module is further configured to input the alignment vector and the attention weight into the deep learning model of the preset network model to obtain deep semantic information; The prediction module is used to predict the deep semantic information through the regression model of the preset network model to obtain the project topic words.

9. A project keyword extraction device, characterized in that, The device includes: a memory, a processor, and a project keyword extraction program stored in the memory and executable on the processor, the project keyword extraction program being configured to implement the steps of the project keyword extraction method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a project keyword extraction program, which, when executed by a processor, implements the steps of the project keyword extraction method as described in any one of claims 1 to 7.

Images