Title generation method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a method, device, electronic device and storage medium for generating a title, the method comprising: obtaining a text to be processed; obtaining at least one input word and input word embedding information corresponding to at least one input word according to the text to be processed; inputting the input word embedding information into a pre-trained title generation model for processing to obtain an output word; the title generation model comprises an encoding layer, an attention layer and a decoding layer; obtaining semantic features of attributes corresponding to the input word based on the input word embedding information through the encoding layer; obtaining an attention vector of the input word based on the semantic features of the attributes corresponding to the input word through the attention layer; extracting at least one output word from at least one input word based on the attention vector through the decoding layer to determine the title of the text to be processed. The embodiment of the present application can generate personalized titles, avoid repeated generation of text and generation of erroneous information, improve the accuracy of the title content, and thus improve the click-through rate of users.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a title generation method, a title generation device, an electronic device and a computer-readable storage medium. Background Art

[0002] On recruitment platforms, a large number of recruitment information are published and updated every day, and users can decide whether to submit an application by browsing the recruitment information. However, the titles of recruitment posts usually have problems such as low quality and homogeneity. For example, many recruitment posts have the same title, which leads to poor user browsing experience and reduces the connection efficiency between the two parties. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a title generation method, a title generation device, an electronic device and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problem, an embodiment of the present invention discloses a title generation method, comprising:

[0005] Get the text to be processed;

[0006] Obtaining at least one input word and input word embedding information corresponding to the at least one input word according to the text to be processed;

[0007] The input word embedding information is input into a pre-trained title generation model for processing to obtain an output word; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to the input word based on the input word embedding information; the attention layer obtains the attention vector of the input word based on the semantic features of the attributes corresponding to the input word; the decoding layer extracts at least one output word from the at least one input word based on the attention vector;

[0008] A title of the to-be-processed text is determined according to the at least one output word.

[0009] Optionally, obtaining at least one input word and input word embedding information corresponding to the at least one input word according to the text to be processed includes:

[0010] Extracting information from the text to be processed to obtain a plurality of attributes and input words corresponding to the plurality of attributes;

[0011] Input word embedding information is obtained according to the input word corresponding attribute and the position information of the input word in all input words corresponding to the input word corresponding attribute.

[0012] Optionally, the encoding layer includes an LSTM unit and an attribute encoding unit; the obtaining of the semantic features of the attributes corresponding to the input word based on the input word embedding information through the encoding layer includes:

[0013] Inputting the input word embedding information into the attribute encoding unit for encoding to obtain input word attribute encoding information;

[0014] The input word attribute encoding information is input into the LSTM unit, and the LSTM unit combines the input word attribute encoding information into the hidden state output at each time step to obtain the overall semantic features of the input word and the semantic features of the corresponding attributes of the input word.

[0015] Optionally, obtaining the attention vector of the input word based on the semantic features of the attribute corresponding to the input word through the attention layer includes:

[0016] Inputting the overall semantic feature of the input word and the semantic feature of the attribute corresponding to the input word into the attention layer to obtain the attention weight of the input word and the attention weight of the attribute corresponding to the input word;

[0017] Determining a dual attention weight of the input word according to the attention weight of the input word and the attention weight of the attribute corresponding to the input word;

[0018] An attention vector of the input word is obtained according to the dual attention weights of the input word and the overall semantic features of the input word.

[0019] Optionally, the decoding layer includes an LSTM unit; and extracting at least one output word from the at least one input word based on the attention vector through the decoding layer includes:

[0020] Inputting the attention vector of the input word and the starting identifier of the text sequence to be processed into the LSTM unit, and the LSTM unit outputs the word with the highest output probability among the at least one input word at the current time step and the hidden state of the current time step;

[0021] Inputting the hidden state outputted at the current time step and the output word at the current time step into the LSTM unit, the LSTM unit outputting the word with the highest output probability among the at least one output word at the next time step and the hidden state at the next time step;

[0022] When the output word of the next time step is the end mark, stop outputting the word with the highest output probability. Optionally, the title generation model is trained in the following manner:

[0023] Acquire training data, wherein the training data includes a text to be processed and a text annotation corresponding to the text to be processed;

[0024] According to the text to be processed, obtaining embedded information of the text to be processed;

[0025] The embedding information of the text to be processed is used as the input of the title generation model; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to each word in the text to be processed based on the embedding information; the attention layer obtains the attention vector of each word based on the semantic features of the attributes corresponding to each word; the decoding layer extracts at least one output word from the training text based on the attention vector;

[0026] Determining output error information according to the output word and the text annotation;

[0027] According to the output error information, the parameters of the title generation model are adjusted to train the title generation model.

[0028] Optionally, the text annotation includes a preset standard output word; and the step of determining the output error information according to the at least one output word and the text annotation includes:

[0029] Compare the output word of the current time step with the preset standard output word to obtain the output error information of the current time step.

[0030] The embodiment of the present invention also discloses a title generation device, the device comprising:

[0031] A text acquisition module is used to acquire the text to be processed;

[0032] An embedding information determination module, configured to obtain at least one input word and input word embedding information corresponding to the at least one input word according to the text to be processed;

[0033] A model processing module, used for inputting the input word embedding information into a pre-trained title generation model for processing to obtain an output word; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to the input word based on the input word embedding information; the attention layer obtains the attention vector of the input word based on the semantic features of the attributes corresponding to the input word; the decoding layer extracts at least one output word from the at least one input word based on the attention vector;

[0034] The title determination module is used to determine the title of the text to be processed according to the at least one output word.

[0035] Optionally, the embedded information determination module includes:

[0036] An information extraction submodule, used to extract information from the text to be processed to obtain a plurality of attributes and input words corresponding to the plurality of attributes;

[0037] The input word embedding information determination submodule is used to obtain the input word embedding information according to the input word corresponding attribute and the position information of the input word in all input words corresponding to the input word corresponding attribute.

[0038] Optionally, the encoding layer includes an LSTM unit and an attribute encoding unit; and the model processing module includes:

[0039] An attribute coding information determination submodule, used for inputting the input word embedding information into the attribute coding unit for coding to obtain input word attribute coding information;

[0040] The semantic feature determination submodule is used to input the input word attribute encoding information into the LSTM unit, and the LSTM unit combines the input word attribute encoding information into the hidden state output at each time step to obtain the overall semantic feature of the input word and the semantic feature of the corresponding attribute of the input word.

[0041] Optionally, the model processing module includes:

[0042] An attention weight determination submodule, used for inputting the overall semantic features of the input word and the semantic features of the attributes corresponding to the input word into the attention layer to obtain the attention weight of the input word and the attention weight of the attributes corresponding to the input word;

[0043] A dual attention weight determination submodule, used to determine the dual attention weight of the input word according to the attention weight of the input word and the attention weight of the attribute corresponding to the input word;

[0044] The attention vector determination submodule is used to obtain the attention vector of the input word according to the dual attention weights of the input word and the overall semantic features of the input word.

[0045] Optionally, the decoding layer includes an LSTM unit; and the model processing module includes:

[0046] A current time step output submodule, used for inputting the attention vector of the input word and the start identifier of the text sequence to be processed into the LSTM unit, and the LSTM unit outputs the word with the highest output probability among the at least one input word at the current time step and the hidden state of the current time step;

[0047] The next time step output submodule is used to input the hidden state outputted at the current time step and the output word at the current time step into the LSTM unit, and the LSTM unit outputs the word with the highest output probability among the at least one output word at the next time step and the hidden state at the next time step;

[0048] The stop submodule is used to stop outputting the word with the highest output probability when the output word in the next time step is the end mark.

[0049] Optionally, the title generation model is trained through the following modules:

[0050] A training data acquisition module, used to acquire training data, wherein the training data includes a text to be processed and a text annotation corresponding to the text to be processed;

[0051] A text processing module, used for obtaining embedded information of the text to be processed according to the text to be processed;

[0052] A model training module, used to use the embedding information of the text to be processed as the input of the title generation model; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to each word in the text to be processed based on the embedding information; the attention layer obtains the attention vector of each word based on the semantic features of the attributes corresponding to each word; the decoding layer extracts at least one output word from the training text based on the attention vector;

[0053] An output error information determination module, used to determine output error information according to the output word and the text annotation;

[0054] The model parameter adjustment module is used to adjust the parameters of the title generation model according to the output error information to train the title generation model.

[0055] Optionally, the text annotation includes preset standard output words; the output error information determination module includes:

[0056] The comparison submodule is used to compare the output word of the current time step with the preset standard output word to obtain the output error information of the current time step.

[0057] An embodiment of the present invention further discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the above-mentioned title generation method when executed by the processor.

[0058] The embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the title generation method described above are implemented.

[0059] The embodiments of the present invention include the following advantages:

[0060] In an embodiment of the present application, input word embedding information can be obtained based on the text to be processed; the input word embedding information is input into a pre-trained title generation model, and the encoding layer of the title generation model can obtain the semantic features of the corresponding attributes of the input word based on the input word embedding information, and the attention layer of the title generation model can obtain the attention vector of the input word based on the semantic features of the corresponding attributes of the input word, thereby combining the corresponding attributes of the input word into the attention vector of the input word; when the decoding layer extracts at least one output word from at least one input word based on the attention vector to determine the title of the text to be processed, compared with directly using the extractive model, the present application combines the corresponding attributes of the input word to determine the title, which can make the generated title more personalized, thereby improving the connection efficiency between the two parties of recruitment and improving recruitment efficiency; compared with directly using the generative model, the embodiment of the present application can avoid the appearance of erroneous information in the generated title and improve the accuracy of the title content. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a flowchart of a method for generating a title provided by an embodiment of the present invention;

[0062] Figure 2 is a flowchart of another method for generating a title provided by an embodiment of the present invention;

[0063] Figure 3 Schematic diagram of an LSTM unit and an attribute encoding unit in an embodiment of the present application;

[0064] Figure 4 It is a flowchart of the steps of the training method of the title generation model in the embodiment of the present application;

[0065] Figure 5 is a schematic diagram of the training process of the title generation model in the embodiment of the present application;

[0066] Figure 6 It is a structural block diagram of a title generating device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] Text generation is an important branch in the field of natural language processing. In recent years, with the continuous development of deep learning technology, text generation technology based on neural networks has received widespread attention and application.

[0069] At present, there are two main methods based on text generation. One is to plan the Pipeline of generating text through grammatical rules, tree rules, etc. The Pipeline method conforms to the process of people editing titles, but there are many processing steps, each step will affect the final result, it is difficult to unify the model, and it is not conducive to rapid iterative optimization.

[0070] The other is a model based on deep learning, mainly represented by generative and extractive methods. The generative method uses the features of the encoding end to complete the sequence generation task at the decoding end, thereby achieving complete end-to-end optimization and completing multimodal tasks; in title generation, the generative summary generates new words and phrases that are not in the original text based on the original text to form a new title. The extractive method extracts a part of the original information as output, and converts it into a variety of different classification tasks at the decoding end through the representation of the encoding end, achieving end-to-end optimization; in title generation, the extractive method needs to extract key sentences and keywords from the original text, and all the generated information comes from the original text.

[0071] Information generated based on generative methods is prone to errors. For example, the generated information does not exist in the original text and is fabricated out of thin air. There are repeated words in the generated information. For example, there is no "commercial insurance" in the recruitment information, but it appears in the generated information. For example, the recruitment information does not involve wages, but the generated information is "recruiting general workers with a monthly salary of over 10,000 yuan." The extractive model mainly copies from the original text, which meets the accuracy requirements of the intelligent title. However, the direct use of the extractive model Pointer Generator Network has a final effect on personalized titles that needs to be improved.

[0072] The core concept of the embodiment of the present invention is that, in the embodiment of the present application, input word embedding information can be obtained according to the text to be processed; the input word embedding information is input into a pre-trained title generation model, and the encoding layer of the title generation model can obtain the semantic features of the corresponding attributes of the input word based on the input word embedding information, and the attention layer of the title generation model obtains the attention vector of the input word based on the semantic features of the corresponding attributes of the input word, thereby combining the corresponding attributes of the input word into the attention vector of the input word; when the decoding layer extracts at least one output word from at least one input word based on the attention vector to determine the title of the text to be processed, compared with directly using the extractive model, the present application combines the corresponding attributes of the input word to determine the title, which can make the generated title more personalized, thereby improving the connection efficiency between the two parties of recruitment and improving the recruitment efficiency; compared with directly using the generative model, the embodiment of the present application can avoid the appearance of erroneous information in the generated title and improve the accuracy of the title content.

[0073] Reference Figure 1 , shows a flowchart of a method for generating a title according to an embodiment of the present invention, wherein the method may specifically include the following steps:

[0074] Step 101, obtaining the text to be processed.

[0075] The title generation method of the embodiment of the present application can be applied to a scenario where a server processes a text to generate a summary corresponding to the text.

[0076] For example, in a recruitment business scenario, a recruitment user can open a recruitment client through a terminal and input a recruitment text into the recruitment client; a job seeker can open a recruitment client through a terminal and browse multiple recruitment posts on a recruitment page. The server can obtain the recruitment text as a to-be-processed text, process the to-be-processed text, and return a title generated for the to-be-processed text. When browsing a recruitment post, the user can obtain key information of the recruitment post by browsing the title of the recruitment post.

[0077] Step 102: obtaining at least one input word and input word embedding information corresponding to the at least one input word according to the text to be processed.

[0078] Input word embedding information can be used to characterize the features of input words. Input words can be obtained from the text to be processed. Based on the text to be processed, multiple input words and input word embedding information corresponding to the multiple input words can be obtained. The input word embedding information can be represented by embedding. Embedding is a way to convert discrete variables into continuous vectors. In deep learning, it has the characteristics of encoding input words with low-dimensional vectors and retaining their features.

[0079] Step 103, inputting the input word embedding information into a pre-trained title generation model for processing to obtain an output word; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to the input word based on the input word embedding information; the attention layer obtains the attention vector of the input word based on the semantic features of the attributes corresponding to the input word; the decoding layer extracts the output word from at least one input word based on the attention vector.

[0080] By inputting the input word embedding information into the pre-trained title generation model, multiple output words can be obtained. The title generation model may include an encoding layer, an attention layer, and a decoding layer. Specifically, the input word embedding information may be input into the encoding layer of the title generation model to obtain the semantic features of the attributes corresponding to the input word; the semantic features of the attributes corresponding to the input word may be input into the attention layer of the title generation model to obtain the attention vector of the input word; the attention vector of the input word may be input into the encoding layer of the title generation model to extract multiple output words from multiple input words.

[0081] Step 104: Determine the title of the text to be processed according to the at least one output word.

[0082] After obtaining multiple output words output by the title generation model, the title of the text to be processed can be determined. For example, the output words are "freight driver" and "housing allowance", and the title of the text to be processed can be "freight driver housing allowance".

[0083] In an embodiment of the present application, input word embedding information can be obtained based on the text to be processed; the input word embedding information is input into the encoding layer, attention layer and decoding layer of the pre-trained title generation model for processing, and the output word can be obtained, so that the title of the text to be processed can be determined. Compared with directly using a generative model, the embodiment of the present application can avoid erroneous information in the generated title and improve the accuracy of the title content; compared with directly using an extractive model, the embodiment of the present application can generate personalized titles, avoid repeated text generation, avoid generating too long or redundant content, and improve user click-through rate.

[0084] Reference Figure 2 , shows a flowchart of an optional embodiment of a title generation method provided in an embodiment of the present application, which may specifically include the following steps:

[0085] Step 201, obtaining the text to be processed.

[0086] In the recruitment business scenario, a recruitment user can open a recruitment client through a terminal and input a recruitment text into the recruitment client; a job seeker can open a recruitment client through a terminal and browse multiple recruitment posts on a recruitment page. The server can obtain the recruitment text as the text to be processed, process the text to be processed, and return the title generated for the text to be processed. When browsing a recruitment post, the user can obtain the key information of the recruitment post by browsing the title of the recruitment post.

[0087] For example, the text to be processed: The cashier intern is responsible for daily income and expenditure management and basic office account verification. Finance major, hard-working fresh graduates, with someone to guide them. 2k during the probation period, 3k + performance bonus + five insurances after becoming a regular employee, accommodation and lunch provided.

[0088] For example, the generated title is: Cashier trainee financial regularization with 3k and five insurances.

[0089] Step 202: Obtain at least one input word and input word embedding information corresponding to the at least one input word according to the text to be processed.

[0090] In an optional embodiment, the step 202 may include the following sub-steps S2021-S2022:

[0091] Sub-step S2021, extracting information from the text to be processed to obtain multiple attributes and input words corresponding to the multiple attributes.

[0092] Information extraction can mean extracting specific information from natural language texts. Through information extraction, the content of natural language texts can be classified. Information extraction can be performed on the text to be processed, and the text to be processed can be classified according to attributes to obtain multiple attributes and input words corresponding to the multiple attributes. For example, in a recruitment business scenario, attributes can be set to age requirements, job skills, job responsibilities, required majors, educational requirements, salary benefits, work experience and other recruitment-related content.

[0093] Multiple attributes and input words corresponding to the multiple attributes can be recorded in the form of a table to obtain structured data, where the first column can record multiple attributes, and the second column can record at least one input word corresponding to the multiple attributes. By querying the data of one row, the attributes of the row and at least one input word corresponding to the attribute can be obtained. Exemplarily, the first row can record: job responsibilities, daily income and expenditure management, basic account verification; the second row can record: salary and benefits, 2k during the probation period, 3k for regularization, performance bonus, five insurances; the third row can record: required major, financial major.

[0094] Those skilled in the art should understand that the above-mentioned attribute settings are merely examples of the present invention, and those skilled in the art may set different attribute types according to actual requirements in different business scenarios, and the present invention is not limited thereto.

[0095] Sub-step S2022, obtaining input word embedding information according to the input word corresponding attribute and the position information of the input word in all input words corresponding to the input word corresponding attribute.

[0096] The input word embedding information may include the input word corresponding attribute and the position information of the input word. The input word embedding information may be obtained according to the input word corresponding attribute and the position information of the input word in at least one input word of the corresponding attribute. The position information of the input word in at least one input word of the corresponding attribute may be represented by a tuple (attribute, number a, number b), wherein number a may be used to represent the position of the input word from front to back in at least one input word of the corresponding attribute, and number b may be used to represent the position of the input word from back to front in at least one input word of the corresponding attribute.

[0097] For example, if in the structured data, the first line records: job responsibilities, daily income and expenditure management, basic account verification; the second line records: salary and benefits, 2k during the probation period, 3k after becoming a regular employee, performance bonus, and five insurances. For the input word "daily income and expenditure management", the corresponding attribute is "job responsibilities". All the input words recorded in the row of "job responsibilities" are "daily income and expenditure management, basic account verification". The input word "daily income and expenditure management" has a position of 1 from the front to the back and a position of 2 from the back to the front in all the input words recorded in this row. The input word embedding information corresponding to the input word "daily income and expenditure management" can be obtained (job responsibilities, 1, 2); for the input word "regularization 3k", the corresponding attribute is "salary and benefits". All the input words recorded in the row of "salary and benefits" are "probation period 2k, regularization 3k, performance award, five insurances". The input word "regularization 3k" has a position of 2 from the front to the back and a position of 3 from the back to the front in all the input words recorded in this row. The input word embedding information corresponding to the input word "regularization 3k" can be obtained (salary and benefits, 2, 3).

[0098] Step 203: Obtain semantic features of attributes corresponding to the input word based on the input word embedding information through the encoding layer.

[0099] In an optional embodiment, the encoding layer may include an LSTM unit and an attribute encoding unit.

[0100] LSTM (Long Short-Term Memory) unit is a neural network that has the ability to memorize long-term and short-term information. LSTM unit can include multiple gating units to control the flow and loss of features. Attribute encoding unit can be used to control the flow of attribute features.

[0101] In an optional embodiment, the step 203 may include the following sub-steps S2031-S2032:

[0102] Sub-step S2031, inputting the input word embedding information into the attribute encoding unit for encoding to obtain input word attribute encoding information.

[0103] The input word embedding information can be input into the attribute encoding unit of the encoding layer for encoding to obtain the input word attribute encoding information.

[0104] Sub-step S2032, inputting the input word attribute encoding information into the LSTM unit, and the LSTM unit combines the input word embedding information into the hidden state output at each time step to obtain the overall semantic features of the input word and the semantic features of the corresponding attributes of the input word.

[0105] The input word attribute encoding information of multiple input words can be input into multiple LSTM units of the encoding layer, and the input word attribute encoding information input at each time step, that is, the encoded input word embedding information, can be combined by the LSTM unit into the hidden state output at each time step. At the last time step, the hidden state output by the last layer of LSTM units can be obtained, and the overall semantic features of the input word and the semantic features of the corresponding attributes of the input word can be determined based on the last layer of hidden states.

[0106] The overall semantic features of the input word can be used to represent the contextual information of the input word. Through the contextual information of the input word, the contextual meaning of the input word, that is, the meaning of the context around the input word, can be obtained, avoiding the problem of inaccurate output probability of the input word caused by only performing downstream tasks based on the semantic features of the corresponding attributes of the input word and ignoring the contextual meaning.

[0107] Through the overall semantic features of the input word, it can be expressed that the input word improves the accuracy of the title content in the context.

[0108] Reference Figure 3 , which is a schematic diagram of an LSTM unit and an attribute encoding unit in an embodiment of the present application. t Can be used to represent the input information of the current time step. C t-1 Can be used to represent the cell state at the previous time step, C t It can be used to represent the cell state at the current time step, which can contain information that requires long-term memory.t-1 Can be used to represent the hidden state of the output of the previous time step, h t It can be used to represent the hidden state of the current time step output. The hidden state can contain information that requires short-term memory.

[0109] The input word sequence x at the current time step can be t , the cell state C at the previous time step t-1 and the hidden state h at the previous time step t-1 Input the word embedding information of the current time step into the attribute encoding unit to obtain the hidden state h output by the LSTM unit at the current time step. t After all input word sequences and input word embedding information are input into the LSTM unit, the hidden state of the last time step can be obtained, that is, the hidden state output by the last layer of LSTM units.

[0110] In an embodiment of the present application, by inputting the input word attribute encoding information output by the attribute encoding unit into the LSTM unit, the input word embedding information of the input word corresponding attribute of the current time step can be controlled to be retained, and the input word embedding information of the input word corresponding attribute that is not the current time step is discarded, so as to purposefully retain the input word embedding information, update the retained input word embedding information to the cell state, obtain the hidden state of the current time step from the updated cell state, and use the hidden state of the current time step as the output of the current time step. When all input word sequences and the corresponding input word embedding information are input into the LSTM unit, the hidden state of the last time step can be obtained as the final output of the encoding layer. At this time, the input word corresponding attribute and context information can be combined into the hidden state of the last time step through encoding, and the overall semantic features of the input word can be obtained through the hidden state of the last time step, which can represent the context information of the input word; the semantic features of the corresponding attributes of the input word can be obtained through the hidden state of the last time step. Downstream tasks can be performed based on the overall semantic features of the input word and the semantic features of the corresponding attributes of the input word.

[0111] Step 204, obtaining an attention vector of the input word based on the semantic features of the attributes corresponding to the input word through the attention layer.

[0112] In an optional embodiment, the step 204 may include the following sub-steps S2041-S2043:

[0113] Sub-step S2041, inputting the overall semantic features of the input word and the semantic features of the attributes corresponding to the input word into the attention layer to obtain the attention weight of the input word and the attention weight of the attributes corresponding to the input word.

[0114] The Coverage Attention mechanism can be set at the attention layer. The overall semantic features of the input word and the semantic features of the attributes corresponding to the input word can be processed using the Coverage Attention mechanism to obtain the attention weight distribution of the input word and the attention weight distribution of the attributes corresponding to the input word.

[0115] In the embodiment of the present application, by adopting the Coverage Attention mechanism, the previous attention weight decision can be used to influence the current attention weight decision, thereby avoiding repeated text generation and playing a role of soft alignment.

[0116] Sub-step S2042, determining the dual attention weight of the input word based on the attention weight of the input word and the attention weight of the attribute corresponding to the input word.

[0117] A Dual Attention mechanism consisting of a local attention mechanism and a global attention mechanism may be set in the attention layer. The attention weight of the input word and the attention weight of the attribute corresponding to the input word may be processed using the DualAttention mechanism. Specifically, the attention weight of the input word and the attention weight of the attribute corresponding to the input word may be weighted and summed to determine the dual attention weight of the input word.

[0118] In an embodiment of the present application, by adopting the Dual Attention mechanism, structured data that records multiple attributes and input words corresponding to the multiple attributes in the form of a table can be used for semantic relevance fusion modeling, thereby achieving an overall description and summary of the generated title, and then effectively combining the attribute information of the input word in the context.

[0119] Sub-step S2043, obtaining the attention vector of the input word according to the dual attention weights of the input word and the overall semantic features of the input word.

[0120] The dual attention weights of the input word and the overall semantic features of the input word can be superimposed and summed to obtain the attention vector of the input word.

[0121] Step 205: extract at least one output word from the at least one input word based on the attention vector through the decoding layer.

[0122] In an optional embodiment, the decoding layer includes an LSTM unit.

[0123] In an optional embodiment, the step 205 may include the following sub-steps S2051-S2053:

[0124] Sub-step S2051, inputting the attention vector of the input word and the starting identifier of the text sequence to be processed into the LSTM unit, and the LSTM unit outputs the word with the highest output probability among the at least one input word in the current time step and the hidden state of the current time step.

[0125] Sub-step S2052, inputting the hidden state output at the current time step and the output word at the current time step into the LSTM unit, and the LSTM unit outputs the word with the highest output probability among the at least one output word at the next time step and the hidden state at the next time step;

[0126] Sub-step S2053, when the output word in the next time step is the end mark, stop outputting the word with the highest output probability.

[0127] An LSTM unit can be set in the decoding layer. The attention vector and the start mark of the input word can be input into the LSTM unit to obtain the output probability of the input word at the current time step, and the word with the highest output probability at the current time step can be output. The word with the highest output probability at the current time step and the hidden state of the output at the current time step can be used as the input of the LSTM unit for the next time step to obtain the output probability of the input word at the next time step, and the word with the highest output probability at the next time step can be output. When the word with the highest output probability at the last time step is the end mark, the word with the highest output probability can no longer be determined.

[0128] Exemplarily, the attention vector of the input word and the starting identifier "start" can be input into the LSTM unit to obtain the input word "freight driver" with the highest output probability in the previous time step, and "freight driver" can be used as the output word in the previous time step. The hidden state output by the LSTM unit in the previous time step and the output word in the previous time step are input into the LSTM unit to obtain the input word "3k+" with the highest output probability in the current time step, and "3k+" can be used as the output word in the current time step. The hidden state output by the LSTM unit in the current time step and the output word in the current time step are input into the LSTM unit to obtain the input word "housing subsidy" with the highest output probability in the next time step, and "housing subsidy" can be used as the output word in the next time step. The hidden state output by the LSTM unit in the next time step and the output word in the next time step are input into the LSTM unit to obtain the word "end" with the highest output probability in the last time step, and the word with the highest output probability is no longer determined at this time.

[0129] At each time step, the output word of the current time step can be extracted from multiple input words, thereby obtaining multiple output words. For example, the input words can be "2k during the probation period, freight driver, 3k+, performance award, five insurances, housing allowance"; the output words can be: "freight driver", "3k+", "housing allowance".

[0130] Step 206: Determine the title of the text to be processed according to the at least one output word.

[0131] Multiple output words obtained by the LSTM unit at the decoding layer end can be concatenated to determine the title of the text to be processed.

[0132] In the embodiment of the present application, according to the text to be processed, the input word embedding information can be obtained; the input word embedding information is input into the pre-trained title generation model, and the encoding layer of the title generation model can obtain the semantic features of the corresponding attributes of the input word based on the input word embedding information, and the attention layer of the title generation model can obtain the attention vector of the input word based on the semantic features of the corresponding attributes of the input word, thereby combining the corresponding attributes of the input word into the attention vector of the input word; when the decoding layer extracts at least one output word from at least one input word based on the attention vector to determine the title of the text to be processed, compared with directly using the extractive model, the present application combines the corresponding attributes of the input word to determine the title, which can make the generated title more personalized, thereby improving the connection efficiency between the two parties of the recruitment and improving the recruitment efficiency; compared with directly using the generative model, the embodiment of the present application can avoid the occurrence of erroneous information in the generated title and improve the accuracy of the title content. By adopting the Coverage Attention mechanism, the previous attention weight decision can be used to influence the current attention weight decision, thereby avoiding repeated generation of text. By adopting the Dual Attention mechanism, the structured data of multiple attributes and input words corresponding to the multiple attributes recorded in the form of a table can be used for semantic relevance fusion modeling, and the attribute information of the input words can be effectively combined in the context.

[0133] Reference Figure 4 As shown, it is a flowchart of the training method of the title generation model in the embodiment of the present application, and the training method of the title generation model includes:

[0134] Step 401: Acquire training data, where the training data includes training text and text annotations corresponding to the training text.

[0135] The text annotation corresponding to the training text can be obtained by manually acquiring the training text, and the text annotation can be used to assist the training model.

[0136] Step 402: Obtain embedding information of the training text according to the training text.

[0137] Step 403, using the embedding information of the training text as the input of the title generation model; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to each word in the training text based on the embedding information; the attention layer obtains the attention vector of each word based on the semantic features of the attributes corresponding to each word; the decoding layer extracts at least one output word from the training text based on the attention vector.

[0138] Step 404: Determine output error information according to the output word and the text annotation.

[0139] In an optional embodiment, the text annotation includes preset standard output words.

[0140] In an optional embodiment, step 404 may include: comparing the output word of the current time step with a preset standard output word to obtain output error information of the current time step.

[0141] Step 405: adjusting the parameters of the title generation model according to the output error information to train the title generation model.

[0142] After obtaining the output error information of the current time step, the error information can be back-propagated to adjust the parameters of the title generation model and the output of the next time step to train the title generation model.

[0143] Reference Figure 5 , which is a schematic diagram of the training process of the title generation model in an embodiment of the present application.

[0144] LSTM network is a special RNN (Recurrent Neural Networks) in the form of a repetitive module chain of a neural network. LSTM can be set at the encoding layer end, and an input word sequence and input word embedding information can be input at each time step. The input word sequence of the current time step, the corresponding attribute information of the input word of the current time step, the hidden state of the previous time step, and the corresponding attribute information of the input word of the previous time step can be input into the LSTM unit to obtain the hidden state of the current time step. Exemplarily, the input word sequence can be: driver / transportation, freight driver, housing subsidy, performance award, C1 license, and the corresponding attributes of the input word can be: required major, job title, welfare benefits, salary benefits, performance bonus, job skills.

[0145] After all input word sequences are input into the LSTM unit for processing, the hidden state of the last time step can be obtained, and the overall semantic features of the input words and the semantic features of the corresponding attributes of the input words can be determined.

[0146] The Coverage Attention mechanism can be used to process the overall semantic features of the input word and the semantic features of the corresponding attributes of the input word to obtain the attention weight distribution of the input word and the attention weight distribution of the corresponding attributes of the input word. The Dual Attention mechanism composed of the local attention mechanism and the global attention mechanism can be used to perform weighted summation of the attention weight of the input word and the attention weight of the corresponding attribute of the input word to obtain the dual attention weight of the input word. The overall semantic features of the input word output by the encoding layer can be obtained, and the dual attention weights of the input word and the overall semantic features of the input word can be superimposed and summed to obtain the attention vector of the input word.

[0147] An LSTM unit can be set at the decoding layer end. The attention vector and the start identifier of the input word can be input into the LSTM unit to obtain the output probability of the input word at the current time step, and the word with the highest output probability at the current time step can be output. The output word at the current time step can be compared with the standard output word preset in the text annotation to determine the output error information, and the parameters of the title generation model can be adjusted according to the output error information. After adjusting the parameters of the title generation model, the word with the highest output probability at the current time step and the hidden state of the current time step can be used as the input of the LSTM unit at the next time step to obtain the output probability of the input word at the next time step, and the word with the highest output probability at the next time step can be output. The output word at the next time step can be compared with the standard output word preset in the text annotation to determine the output error information, and the parameters of the title generation model can be adjusted according to the output error information. By continuously adjusting the parameters of the title generation model, the title generation model can be trained.

[0148] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0149] Reference Figure 6 , shows a structural block diagram of a title generation device provided by an embodiment of the present invention, which may specifically include the following modules:

[0150] The text acquisition module 601 is used to acquire the text to be processed.

[0151] The embedding information determination module 602 is used to obtain at least one input word and input word embedding information corresponding to the at least one input word according to the text to be processed.

[0152] The model processing module 603 is used to input the input word embedding information into a pre-trained title generation model for processing to obtain an output word; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to the input word based on the input word embedding information; the attention layer obtains the attention vector of the input word based on the semantic features of the attributes corresponding to the input word; the decoding layer extracts at least one output word from the at least one input word based on the attention vector.

[0153] The title determination module 604 is used to determine the title of the text to be processed according to the at least one output word.

[0154] In an embodiment of the present invention, the embedded information determination module includes:

[0155] The information extraction submodule is used to extract information from the text to be processed to obtain multiple attributes and input words corresponding to the multiple attributes.

[0156] The input word embedding information determination submodule is used to obtain the input word embedding information according to the input word corresponding attribute and the position information of the input word in all input words corresponding to the input word corresponding attribute.

[0157] In the embodiment of the present invention, the encoding layer includes an LSTM unit and an attribute encoding unit; the model processing module includes:

[0158] The attribute coding information determination submodule is used to input the input word embedding information into the attribute coding unit for encoding to obtain the input word attribute coding information.

[0159] The semantic feature determination submodule is used to input the input word attribute encoding information into the LSTM unit, and the LSTM unit combines the input word attribute encoding information into the hidden state output at each time step to obtain the overall semantic feature of the input word and the semantic feature of the corresponding attribute of the input word.

[0160] In an embodiment of the present invention, the model processing module includes:

[0161] The attention weight determination submodule is used to input the overall semantic features of the input word and the semantic features of the attributes corresponding to the input word into the attention layer to obtain the attention weight of the input word and the attention weight of the attributes corresponding to the input word.

[0162] The dual attention weight determination submodule is used to determine the dual attention weight of the input word based on the attention weight of the input word and the attention weight of the attribute corresponding to the input word.

[0163] The attention vector determination submodule is used to obtain the attention vector of the input word according to the dual attention weights of the input word and the overall semantic features of the input word.

[0164] In an embodiment of the present invention, the decoding layer includes an LSTM unit; the model processing module includes:

[0165] The current time step output submodule is used to input the attention vector of the input word and the starting identifier of the text sequence to be processed into the LSTM unit, and the LSTM unit outputs the word with the highest output probability among the at least one input word in the current time step and the hidden state of the current time step.

[0166] The next time step output submodule is used to input the hidden state of the current time step output and the output word of the current time step into the LSTM unit, and the LSTM unit outputs the word with the highest output probability among the at least one output word in the next time step and the hidden state of the next time step.

[0167] The stop submodule is used to stop outputting the word with the highest output probability when the output word of the next time step is the end mark

[0168] In the embodiment of the present invention, the title generation model is trained by the following modules:

[0169] The training data acquisition module is used to acquire training data, wherein the training data includes the text to be processed and the text annotation corresponding to the text to be processed.

[0170] The text processing module is used to obtain the embedded information of the text to be processed according to the text to be processed.

[0171] A model training module is used to use the embedding information of the text to be processed as the input of the title generation model; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the corresponding attributes of each word in the text to be processed based on the embedding information; the attention layer obtains the attention vector of each word based on the semantic features of the corresponding attributes of each word; the decoding layer extracts at least one output word from the training text based on the attention vector.

[0172] The output error information determination module is used to determine the output error information according to the output word and the text annotation.

[0173] The model parameter adjustment module is used to adjust the parameters of the title generation model according to the output error information to train the title generation model.

[0174] In an embodiment of the present invention, the text annotation includes a preset standard output word; the output error information determination module includes:

[0175] The comparison submodule is used to compare the output word of the current time step with the preset standard output word to obtain the output error information of the current time step.

[0176] In an embodiment of the present application, input word embedding information can be obtained based on the text to be processed; the input word embedding information is input into a pre-trained title generation model, and the encoding layer of the title generation model can obtain the semantic features of the corresponding attributes of the input word based on the input word embedding information, and the attention layer of the title generation model can obtain the attention vector of the input word based on the semantic features of the corresponding attributes of the input word, thereby combining the corresponding attributes of the input word into the attention vector of the input word; when the decoding layer extracts at least one output word from at least one input word based on the attention vector to determine the title of the text to be processed, compared with directly using the extractive model, the present application combines the corresponding attributes of the input word to determine the title, which can make the generated title more personalized, thereby improving the connection efficiency between the two parties of recruitment and improving recruitment efficiency; compared with directly using the generative model, the embodiment of the present application can avoid the appearance of erroneous information in the generated title and improve the accuracy of the title content.

[0177] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0178] An embodiment of the present invention further provides an electronic device, including:

[0179] The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, each process of the above-mentioned title generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0180] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned title generation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0181] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0182] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0183] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0184] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0186] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0187] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0188] The above is a detailed introduction to a title generation method, device, electronic device and storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for generating a title, characterized in that: include: Get the text to be processed; Obtaining at least one input word and input word embedding information corresponding to the at least one input word according to the text to be processed; Inputting the input word embedding information into a pre-trained title generation model for processing to obtain an output word; The title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to the input words based on the input word embedding information; the attention layer obtains the attention vector of the input words based on the semantic features of the attributes corresponding to the input words; Extracting at least one output word from the at least one input word based on the attention vector through the decoding layer; Determining a title of the text to be processed according to the at least one output word; The obtaining, according to the text to be processed, at least one input word and input word embedding information corresponding to the at least one input word comprises: Extracting information from the text to be processed to obtain a plurality of attributes and input words corresponding to the plurality of attributes; Input word embedding information is obtained according to the input word corresponding attribute and the position information of the input word in all input words corresponding to the input word corresponding attribute.

2. The method according to claim 1, characterized in that The encoding layer includes an LSTM unit and an attribute encoding unit; the semantic features of the attributes corresponding to the input words are obtained through the encoding layer based on the input word embedding information, including: Inputting the input word embedding information into the attribute encoding unit for encoding to obtain input word attribute encoding information; The input word attribute encoding information is input into the LSTM unit, and the LSTM unit combines the input word attribute encoding information into the hidden state output at each time step to obtain the overall semantic features of the input word and the semantic features of the corresponding attributes of the input word.

3. The method according to claim 2, characterized in that The step of obtaining the attention vector of the input word based on the semantic features of the attribute corresponding to the input word through the attention layer includes: Inputting the overall semantic feature of the input word and the semantic feature of the attribute corresponding to the input word into the attention layer to obtain the attention weight of the input word and the attention weight of the attribute corresponding to the input word; Determining a dual attention weight of the input word according to the attention weight of the input word and the attention weight of the attribute corresponding to the input word; An attention vector of the input word is obtained according to the dual attention weights of the input word and the overall semantic features of the input word.

4. The method according to claim 3, characterized in that The decoding layer includes an LSTM unit; the step of extracting at least one output word from the at least one input word based on the attention vector through the decoding layer includes: Inputting the attention vector of the input word and the starting identifier of the text sequence to be processed into the LSTM unit, the LSTM unit outputting the word with the highest output probability among the at least one input word at the current time step and the hidden state at the current time step; Inputting the hidden state outputted at the current time step and the output word at the current time step into the LSTM unit, the LSTM unit outputting the word with the highest output probability among the at least one output word at the next time step and the hidden state at the next time step; When the output word of the next time step is the end token, stop outputting the word with the highest output probability.

5. The method according to claim 1, characterized in that The title generation model is trained as follows: Acquire training data, wherein the training data includes training text and text annotations corresponding to the training text; According to the training text, obtaining embedding information of the training text; The embedding information of the training text is used as the input of the title generation model; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to each word in the training text based on the embedding information; the attention layer obtains the attention vector of each word based on the semantic features of the attributes corresponding to each word; the decoding layer extracts at least one output word from the training text based on the attention vector; Determining output error information according to the at least one output word and the text annotation; According to the output error information, the parameters of the title generation model are adjusted to train the title generation model.

6. The method according to claim 5, characterized in that The decoding layer includes an LSTM unit; the text annotation includes a preset standard output word; and the output error information is determined according to the at least one output word and the text annotation, including: The output word of the LSTM unit at the current time step is compared with the preset standard output word to obtain the output error information of the current time step.

7. A title generating device, characterized in that: The device comprises: A text acquisition module is used to acquire the text to be processed; An embedding information determination module, configured to obtain at least one input word and input word embedding information corresponding to the at least one input word according to the text to be processed; A model processing module, used for inputting the input word embedding information into a pre-trained title generation model for processing to obtain an output word; the title generation model includes an encoding layer, an attention layer and a decoding layer; the encoding layer obtains the semantic features of the attributes corresponding to the input word based on the input word embedding information; the attention layer obtains the attention vector of the input word based on the semantic features of the attributes corresponding to the input word; the decoding layer extracts at least one output word from the at least one input word based on the attention vector; A title determination module, used to determine the title of the text to be processed according to the at least one output word; The embedded information determination module includes: An information extraction submodule, used to extract information from the text to be processed to obtain a plurality of attributes and input words corresponding to the plurality of attributes; The input word embedding information determination submodule is used to obtain the input word embedding information according to the input word corresponding attribute and the position information of the input word in all input words corresponding to the input word corresponding attribute.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the title generation method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the title generation method according to any one of claims 1 to 6 are implemented.

Images