Method, device, equipment and storage medium for generating embedding vector of digital text

Abstract

The present application discloses a method, device, equipment and storage medium for generating an embedding vector of a digital text, and the present application belongs to the field of artificial intelligence technology. The present application converts the character string of the digital text in the training corpus, and uses the training corpus to construct numerical feature samples and arithmetic feature samples, converts the above samples into word vectors, obtains an initial embedding vector, and uses the initial embedding vector to iteratively train the joint neural network model, obtains the model weight average, and obtains the embedding vector of the digital text. The present application converts the digital text into a Chinese character string, and then converts the Chinese character string into a word vector, retains the digital text features, and combines the arithmetic features in the training corpus to jointly train the neural network, so that the generated embedding vector of the digital text has the ability to compare the size of numbers and perform arithmetic operations, thereby coping with more complex downstream processing tasks while improving the accuracy of model operations.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a method, device, equipment and storage medium for generating an embedding vector of digital text. Background Art

[0002] Word vectors have become a basic concept in the field of natural language processing. Technicians map words or phrases from the vocabulary into word vectors (i.e., continuous real number vectors), then input them into various machine learning models, optimize and train the models through loss functions, and then use the trained models for specific tasks such as text classification, machine reading, and text summarization. Compared with traditional discrete representations, such as TF-IDF and bag-of-words models, the effects of tasks using word vectors have been greatly improved because continuous vectors contain richer features.

[0003] However, the existing technology generally uses a relatively rough method to process a special type of symbol in the text, namely numbers, which are generally filtered or simply used as identifiers. For example, patent CN106021272B discloses a keyword automatic extraction method based on distributed expression word vector calculation, in which numbers are directly filtered and not processed; while other methods either uniformly treat numbers as special symbols, namely UNK symbols; or simply perform structural splitting on numbers, for example, splitting the number "1234" into "1-234", "12-34" or "123-4", and assigning word vectors to numbers from 0 to 9, so that the word vectors corresponding to various numbers can be spliced ​​together by the above method.

[0004] In the existing text vector generation scheme, the processing method for numerical text fails to fully capture the semantic information contained in the numeric symbols appearing in the context. In the downstream natural language processing tasks involving numbers, many papers have proved that such processing methods are not effective. For example, machine reading tasks involving complex digital calculations require the model to learn to compare the size of numbers to some extent. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a method, apparatus, computer device and storage medium for generating an embedding vector of digital text, so as to solve the technical problem that the existing text vector generation filters numbers or simply treats them as identifiers, and fails to fully capture the semantic information in the context of numerical text.

[0006] In order to solve the above technical problems, the embodiment of the present application provides a method for generating an embedding vector of a digital text, which adopts the following technical solution:

[0007] A method for generating an embedding vector of digital text, comprising:

[0008] Obtaining a training corpus and preprocessing the training corpus, wherein the preprocessing includes converting digital text in the training corpus into Chinese character strings;

[0009] Constructing a feature sample set using the preprocessed training corpus, wherein the feature sample set includes numerical feature samples and arithmetic feature samples;

[0010] Convert the numerical feature samples into word vectors to obtain the initial numerical feature embedding vector;

[0011] Perform word vector conversion on the arithmetic feature samples to obtain the initial arithmetic feature embedding vector;

[0012] Iteratively train the preset joint neural network model using the initial numerical feature embedding vector and the initial arithmetic feature embedding vector until the model is fitted;

[0013] Get the weighted average of the joint neural network model after model fitting to get the embedding vector of the digital text.

[0014] Furthermore, the steps of obtaining a training corpus and preprocessing the training corpus specifically include:

[0015] Get the training corpus and search for digital text in the training corpus;

[0016] Get the preset digital unit set;

[0017] The digital text in the training corpus is converted into a character string according to the digital unit set to obtain a numerical text corresponding to the digital text, wherein the numerical text is a Chinese character string corresponding to the digital text.

[0018] Furthermore, the step of constructing a feature sample set using the preprocessed training corpus specifically includes:

[0019] Obtain the numerical corpus in the training corpus and construct numerical feature samples based on the numerical corpus;

[0020] Obtain arithmetic corpus from the training corpus and construct arithmetic feature samples based on the arithmetic corpus.

[0021] Furthermore, the steps of obtaining the numerical corpus in the training corpus set and constructing the numerical feature sample based on the numerical corpus specifically include:

[0022] Traverse the numerical corpus, obtain all numerical texts in the numerical corpus, and build a numerical text set;

[0023] Get a preset sliding window and use it to slide in the numerical text set to generate numerical feature samples.

[0024] Furthermore, the steps of converting the numerical feature samples into word vectors to obtain the initial numerical feature embedding vectors specifically include:

[0025] Acquire all numerical texts in the numerical feature sample to obtain a first numerical text;

[0026] Convert each Chinese character of the first numerical text into a word vector in sequence to obtain a first initial embedding vector, wherein each Chinese character of the first numerical text corresponds to a first initial embedding vector;

[0027] Count the frequency coefficient of each Chinese character;

[0028] An initial numerical feature embedding vector is constructed based on the first initial embedding vector and the frequency coefficient of each Chinese character.

[0029] Furthermore, the steps of obtaining arithmetic corpus in the training corpus and constructing arithmetic feature samples based on the arithmetic corpus specifically include:

[0030] Traverse the arithmetic data and obtain the arithmetic text in the arithmetic data;

[0031] Perform semantic recognition on arithmetic text to obtain the operation symbols of the arithmetic text;

[0032] Extracting the numerical text of the arithmetic text to obtain a second numerical text;

[0033] An arithmetic feature sample is constructed based on the second numerical text and the operation symbol of the arithmetic text.

[0034] Furthermore, the step of converting the arithmetic feature samples into word vectors to obtain the initial arithmetic feature embedding vectors specifically includes:

[0035] Convert each Chinese character of the second numerical text into a word vector in sequence to obtain a second initial embedding vector, wherein each Chinese character of the second numerical text corresponds to a second initial embedding vector;

[0036] Convert the operator symbol into a word vector to obtain the operator embedding vector;

[0037] Based on the second initial embedding vector and the operator embedding vector, an initial arithmetic feature embedding vector is constructed.

[0038] Furthermore, the joint neural network model includes a first neural network model and a second neural network model, and the preset joint neural network model is iteratively trained using the initial numerical feature embedding vector and the initial arithmetic feature embedding vector until the model is fitted, specifically including:

[0039] Inputting the initial numerical feature embedding vector into the first neural network model, and iteratively updating the first neural network model using a preset cross entropy loss function;

[0040] Inputting the initial arithmetic feature embedding vector into the second neural network model, and iteratively updating the second neural network model using a preset mean square error loss function;

[0041] Obtaining the gradient value of the first neural network model after the iteration is completed, to obtain a first gradient value;

[0042] Obtaining the gradient value of the second neural network model after the iteration is completed to obtain a second gradient value;

[0043] The gradient average of the first gradient value and the second gradient value is calculated, and the joint neural network model is optimized using the gradient average to obtain a fitted joint neural network model.

[0044] In order to solve the above technical problems, the embodiment of the present application also provides a device for generating an embedding vector of a digital text, which adopts the following technical solution:

[0045] A device for generating an embedding vector of digital text, comprising:

[0046] A corpus preprocessing module is used to obtain a training corpus set and preprocess the training corpus set, wherein the preprocessing includes converting digital text in the training corpus set into Chinese character strings;

[0047] A sample set construction module is used to construct a feature sample set using the preprocessed training corpus, wherein the feature sample set includes numerical feature samples and arithmetic feature samples;

[0048] The first vector conversion module is used to convert the numerical feature samples into word vectors to obtain the initial numerical feature embedding vector;

[0049] The second vector conversion module is used to convert the arithmetic feature samples into word vectors to obtain an initial arithmetic feature embedding vector;

[0050] A joint training module, used for iteratively training a preset joint neural network model using an initial numerical feature embedding vector and an initial arithmetic feature embedding vector until the model is fitted;

[0051] The embedding vector generation module is used to obtain the weighted average value of the joint neural network model after model fitting and the embedding vector of the digital text.

[0052] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0053] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for generating an embedding vector of digital text as described in any one of the above items are implemented.

[0054] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0055] A computer-readable storage medium having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by a processor, implement the steps of the method for generating an embedding vector of digital text as described in any one of the above.

[0056] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0057] The present application discloses a method, device, equipment and storage medium for generating an embedding vector of a digital text, and the present application belongs to the field of artificial intelligence technology. The present application converts the character string of the digital text in the training corpus, and uses the training corpus that has completed the character string conversion to construct numerical feature samples and arithmetic feature samples, converts the numerical feature samples and arithmetic feature samples into word vectors, obtains the initial numerical feature embedding vector and the initial arithmetic feature embedding vector, and uses the above two initial embedding vectors to iteratively train the joint neural network model, obtains the average value of the model weight after the model is fitted, and obtains the embedding vector of the digital text. The present application converts the digital text into a Chinese character string, and then converts the Chinese character string into a word vector, retains the digital text features, and uses the arithmetic features in the training corpus to jointly train the neural network, so that the generated digital embedding vector has the ability to compare the size of numbers and perform arithmetic operations, thereby coping with more complex downstream processing tasks while improving the accuracy of model operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 An exemplary system architecture diagram is shown in which the present application can be applied;

[0060] Figure 2 A flowchart of an embodiment of a method for generating an embedding vector of a digital text according to the present application is shown;

[0061] Figure 3A schematic diagram of the structure of an embodiment of a device for generating an embedding vector of a digital text according to the present application is shown;

[0062] Figure 4 A schematic structural diagram of an embodiment of a computer device according to the present application is shown. DETAILED DESCRIPTION

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0064] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0065] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0066] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0067] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0068] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.

[0069] Server 105 can be a server that provides various services, such as a background server that provides support for pages displayed on terminal devices 101, 102, and 103. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0070] It should be noted that the method for generating an embedding vector of a digital text provided in the embodiment of the present application is generally executed by a server, and accordingly, the device for generating an embedding vector of a digital text is generally arranged in the server.

[0071] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0072] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for generating an embedding vector of a digital text according to the present application. The present application embodiment can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0073] The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning. The method for generating an embedding vector of digital text includes the following steps:

[0074] S201. Obtain a training corpus and preprocess the training corpus. Herein, the preprocessing includes converting digital texts in the training corpus into Chinese character strings.

[0075] Herein, the training corpus of the present application adopts the DROP (Discrete Reasoning Over Paragraphs) dataset. There are approximately 77k questions in the DROP training set and approximately 9.5k questions in the DROP development set.

[0076] Specifically, the server obtains the training corpus from the DROP dataset, extracts all digital texts in the training corpus using regular expressions, and performs string conversion on all digital texts, that is, converting digital texts into Chinese character strings. For example, the digital text "123" is converted into the Chinese character string "one hundred and twenty-three".

[0077] S202. Construct a feature sample set using the preprocessed training corpus. Herein, the feature sample set includes numerical feature samples and arithmetic feature samples.

[0078] Specifically, after the server completes the string conversion of all digital texts, it constructs a feature sample set using the training corpus after the string conversion. Herein, the feature sample set includes numerical feature samples and arithmetic feature samples. The server obtains the corpus carrying numerical texts in the training corpus and constructs a numerical text set using these. Then, a preset sliding window is used to slide in the numerical text set, and a numerical sequence is generated each time it slides. This numerical sequence is a numerical feature sample. The server obtains the corpus carrying arithmetic processes in the training corpus and extracts the numerical texts and operation symbols in the arithmetic process corpus to construct arithmetic feature samples.

[0079] S203. Perform word vector conversion on the numerical feature samples to obtain initial numerical feature embedding vectors.

[0080] S204. Perform word vector conversion on the arithmetic feature samples to obtain initial arithmetic feature embedding vectors.

[0081] Herein, the present application performs word vector conversion on the feature samples based on the Word2vec bag-of-words model. Word2vec is a group of related models used to generate word vectors. These models are shallow and two-layer neural networks used to train to reconstruct linguistic word texts. The network is represented by words and needs to guess the input words at adjacent positions. Under the assumption of the bag-of-words model in word2vec, the order of words is not important. After training is completed, the word2vec model can be used to map each word to a vector, which can be used to represent the relationship between words. This vector is the hidden layer of the neural network.

[0082] Specifically, the present application performs word vector conversion on the numerical text strings and operation symbol strings in the numerical feature samples and the arithmetic feature samples through the Word2vec bag-of-words model to obtain the initial numerical feature embedding vector and the initial arithmetic feature embedding vector.

[0083] S205, iteratively training the preset joint neural network model using the initial numerical feature embedding vector and the initial arithmetic feature embedding vector until the model is fitted.

[0084] Specifically, the joint neural network model of the present application adopts a dual model structure, namely a numerical model and an arithmetic model. The numerical model is used to compare numerical values, and the arithmetic model is used for arithmetic operations. The initial numerical feature embedding vector is input into the numerical model for iterative training, and the initial arithmetic feature embedding vector is input into the arithmetic model for iterative training. Then, the gradient values ​​of the numerical model and the arithmetic model are obtained respectively, and the joint neural network model is optimized by using the gradient average values ​​of the numerical model and the arithmetic model to make the joint neural network model fit.

[0085] S206, obtaining the weighted average value of the joint neural network model after model fitting, and obtaining the embedding vector of the digital text.

[0086] Specifically, after fitting the joint neural network model, the weight parameters of each network layer in the numerical model and the arithmetic model are obtained, and the average value of the weight parameters of each network layer is calculated. The average values ​​of the weight parameters of each network layer are combined to obtain the embedding vector corresponding to the digital text in the training corpus.

[0087] In the above embodiment, the present application retains the characteristics of the digital text by converting the digital text into a Chinese character string and then converting the Chinese character string into a word vector, and uses the arithmetic features in the training corpus to jointly train the neural network, so that the generated digital embedding vector has the ability to compare the size of numbers and perform arithmetic operations, thereby coping with more complex downstream processing tasks while improving the accuracy of model operations.

[0088] In a specific embodiment of the present application, the steps of obtaining a training corpus and preprocessing the training corpus specifically include:

[0089] Get the training corpus and search for digital text in the training corpus;

[0090] Get the preset digital unit set;

[0091] The digital text in the training corpus is converted into a character string according to the digital unit set to obtain a numerical text corresponding to the digital text, wherein the numerical text is a Chinese character string corresponding to the digital text.

[0092] Specifically, the server extracts all numerical texts in the training corpus using regular expressions, identifies the numerical magnitudes of the numerical texts, and obtains a preset set of numerical units. The set of numerical units is, for example, [..., hundred million, ten million, million, hundred thousand, ten thousand, thousand, hundred, ten, nine, eight, seven, six, five, four, three, two, one, zero]. According to the numerical values of the numerical texts and the set of numerical units, the numerical texts are converted into corresponding Chinese character strings.

[0093] For example, for the numerical text "450,000", the largest unit is "hundred thousand", and the numerical value corresponding to the units digit is "0". Therefore, when converting the numerical text "450,000" into a character string, the resulting Chinese character string is "four hundred thousand fifty thousand zero thousand zero hundred zero ten zero".

[0094] It should be noted that when there is a decimal point in the numerical text, the decimal point symbol "." is converted into the Chinese character "point". For example, "123.45" is converted into "one hundred twenty-three point forty-five".

[0095] In a specific embodiment of the present application, the step of constructing a feature sample set using the preprocessed training corpus specifically includes:

[0096] Obtain the numerical corpus in the training corpus and construct numerical feature samples based on the numerical corpus;

[0097] Obtain the arithmetic corpus in the training corpus and construct arithmetic feature samples based on the arithmetic corpus.

[0098] In a specific embodiment of the present application, the step of obtaining the numerical corpus in the training corpus and constructing numerical feature samples based on the numerical corpus specifically includes:

[0099] Traverse the numerical corpus to obtain all numerical texts in the numerical corpus and construct a numerical text set;

[0100] Obtain a preset sliding window and use the sliding window to slide in the numerical text set to generate numerical feature samples.

[0101] In a specific embodiment of the present application, the step of obtaining the arithmetic corpus in the training corpus and constructing arithmetic feature samples based on the arithmetic corpus specifically includes:

[0102] Traverse the arithmetic corpus to obtain the arithmetic texts in the arithmetic corpus;

[0103] Perform semantic recognition on the arithmetic texts to obtain the operation symbols of the arithmetic texts;

[0104] Extract the numerical texts of the arithmetic texts to obtain second numerical texts;

[0105] Based on the second numerical texts and the operation symbols of the arithmetic texts, construct arithmetic feature samples.

[0106] Specifically, in a specific embodiment of the present application, after completing the string conversion of the digital text, the server obtains the corpus carrying numerical text in the training corpus, obtains the numerical corpus, traverses the numerical corpus, obtains all numerical texts in the numerical corpus, constructs a numerical text set, obtains a preset sliding window, and uses the sliding window to slide in the numerical text set to generate a numerical feature sample.

[0107] For example, for any target numerical text X in the numerical text set, set the sliding window size to 2C, where C is the number of numerical texts, select C numerical texts with the smallest difference with X and less than X, and C numerical texts with a small difference with X and greater than X, to form a numerical feature sample, and configure a label for the numerical feature sample, the label is the size rank corresponding to the numerical text. Take the numerical text "five hundred and ten" as an example, let C = 2, assuming that in the numerical text set, there are two numerical texts "two hundred and thirty-zero" and "three hundred and forty-five" that are less than "five hundred and ten" and have the closest difference with "five hundred and ten", and at the same time, "five hundred and fifty-six" and "five hundred and fifty-six" are two numerical values ​​that are greater than "five hundred and ten" and have the closest difference with "five hundred and ten", then a sample [two hundred and thirty-zero, three hundred and forty-five, five hundred and ten, five hundred and fifty-six, six hundred and fifty-zero] centered on the string expression of "five hundred and fifty-six" is formed, and the sample labels are [1, 2, 3, 4, 5]. Referring to the above process, for the numerical texts in the entire numerical text set, a numerical feature sample with an element of 2C+1 is generated, except for the largest 2 numerical texts and the smallest 2 numerical texts in the numerical text set. For these 4 numerical texts, the server randomly selects other numerical texts to form the numerical feature samples.

[0108] In another specific embodiment of the present application, after completing the string conversion of the digital text, the server obtains the corpus carrying arithmetic text in the training corpus, traverses the arithmetic corpus, obtains the arithmetic text in the arithmetic corpus, performs semantic recognition on the arithmetic text, obtains the operation symbols of the arithmetic text, extracts the numerical text of the arithmetic text, obtains the second numerical text, and constructs an arithmetic feature sample based on the second numerical text and the operation symbols of the arithmetic text.

[0109] For example, in an arithmetic term, "Xiao Ming picked twenty apples in the morning and thirty peaches in the afternoon. How many fruits did Xiao Ming pick in total? Answer: twenty plus thirty equals fifty", the numerical text of the arithmetic text includes "twenty", "thirty" and "fifty", and the operation symbols include "plus" and "equal", so the arithmetic feature sample is expressed as [twenty, plus, thirty, equal, fifty], and the correct answer of the arithmetic feature sample "fifty" is marked.

[0110] In a specific embodiment of the present application, the step of converting the numerical feature sample into a word vector to obtain an initial numerical feature embedding vector specifically includes:

[0111] Acquire all numerical texts in the numerical feature sample to obtain a first numerical text;

[0112] Convert each Chinese character of the first numerical text into a word vector in sequence to obtain a first initial embedding vector, wherein each Chinese character of the first numerical text corresponds to a first initial embedding vector;

[0113] Count the frequency coefficient of each Chinese character;

[0114] An initial numerical feature embedding vector is constructed based on the first initial embedding vector and the frequency coefficient of each Chinese character.

[0115] Specifically, each numerical feature sample includes several numerical texts. Based on the Word2vec bag-of-words model, each Chinese character in each numerical text of the numerical feature sample is converted into a word vector to obtain a first initial embedding vector, and the frequency coefficient of each Chinese character is counted, wherein the frequency coefficient of each Chinese character in the numerical text is equal to the number of times the Chinese character appears in the entire training corpus / the number of numerical feature samples.

[0116] The first initial embedding vector of each Chinese character in the numerical text is multiplied by the corresponding frequency coefficient and then the vectors are summed up and the average vector is calculated to obtain an initial embedding vector of the numerical text. The initial embedding vectors of all numerical texts are summed up and the average vector is calculated to obtain the initial numerical feature embedding vector.

[0117] In a specific embodiment of the present application, the step of converting the arithmetic feature sample into a word vector to obtain an initial arithmetic feature embedding vector specifically includes:

[0118] Convert each Chinese character of the second numerical text into a word vector in sequence to obtain a second initial embedding vector, wherein each Chinese character of the second numerical text corresponds to a second initial embedding vector;

[0119] Convert the operator symbol into a word vector to obtain the operator embedding vector;

[0120] Based on the second initial embedding vector and the operator embedding vector, an initial arithmetic feature embedding vector is constructed.

[0121] Specifically, based on the Word2vec bag-of-words model, each Chinese character of the second numerical text is converted into a word vector in turn to obtain a second initial embedding vector, and the operator symbol is converted into a word vector to obtain an operator embedding vector. The second initial embedding vector and the operator embedding vector are summed, and the average vector is calculated, the initial arithmetic feature embedding vector.

[0122] In a specific embodiment of the present application, the joint neural network model includes a first neural network model and a second neural network model, and the preset joint neural network model is iteratively trained using an initial numerical feature embedding vector and an initial arithmetic feature embedding vector until the model is fitted, specifically including:

[0123] Inputting the initial numerical feature embedding vector into the first neural network model, and iteratively updating the first neural network model using a preset cross entropy loss function;

[0124] Inputting the initial arithmetic feature embedding vector into the second neural network model, and iteratively updating the second neural network model using a preset mean square error loss function;

[0125] Obtaining the gradient value of the first neural network model after the iteration is completed, to obtain a first gradient value;

[0126] Obtaining the gradient value of the second neural network model after the iteration is completed to obtain a second gradient value;

[0127] The gradient average of the first gradient value and the second gradient value is calculated, and the joint neural network model is optimized using the gradient average to obtain a fitted joint neural network model.

[0128] Specifically, a joint neural network model of two neural network architectures is constructed, including a first neural network model and a second neural network model. The first neural network is used to train to acquire the ability to compare numerical values. By defining its input layer dimension as N, that is, the dimension of the initial numerical feature embedding vector, at the beginning of each training cycle, the initial numerical feature embedding vector of the numerical feature sample of each input batch is input into the input layer of the neural network, and the output layer dimension is defined as the total number of all numerical texts in the numerical feature sample, and the dimension is set to V. The softmax function is then used to output the label corresponding to each numerical text in each numerical feature sample, and the cross entropy loss function is used to optimize the first neural network model.

[0129] The second neural network is used to train the ability to perform arithmetic operations. The dimension of its input layer is defined as N, that is, the dimension of the initial arithmetic feature embedding vector. Its output layer is defined as the sample labels with correct operations corresponding to the initial arithmetic feature embedding vector. The mean square error loss function is used to optimize the second neural network model.

[0130] The first neural network model and the second neural network model are iteratively trained simultaneously, and the losses are calculated respectively using the above-mentioned loss function, and the gradients of the two neural network models are updated, the gradient value of the first neural network model after the iteration is completed is obtained, and the first gradient value is obtained, the gradient value of the second neural network model after the iteration is completed is obtained, and the second gradient value is obtained, and the gradient average of the first gradient value and the second gradient value is calculated, and the gradient average is used to optimize the joint neural network model to obtain a fitted joint neural network model.

[0131] It should be noted that, during each optimization, the first neural network calculates its own gradient value and optimizes the parameters of its own neural network, and passes the gradient value to the second neural network. The second neural network calculates its own gradient value, adds it to the received gradient value of the first neural network, and calculates the gradient average value. The gradient average value is used to continue to optimize the parameters of the joint neural network. When the two neural network models are fully trained, the average weight parameters of each network layer of the two neural network models are obtained, and the average weight parameters of each network layer of the two neural network models are combined to obtain the embedding vector of the numerical text.

[0132] In this embodiment, the present application discloses a method for generating an embedding vector of a digital text, and the present application belongs to the field of artificial intelligence technology. The present application converts the character string of the digital text in the training corpus, and uses the training corpus that completes the character string conversion to construct numerical feature samples and arithmetic feature samples, converts the numerical feature samples and arithmetic feature samples into word vectors, obtains the initial numerical feature embedding vector and the initial arithmetic feature embedding vector, and uses the above two initial embedding vectors to iteratively train the joint neural network model, obtains the model weight average after model fitting, and obtains the embedding vector of the digital text. The present application converts the digital text into a Chinese character string, and then converts the Chinese character string into a word vector, retains the digital text features, and uses the arithmetic features in the training corpus to jointly train the neural network, so that the generated digital embedding vector has the ability to compare the size of numbers and perform arithmetic operations, thereby coping with more complex downstream processing tasks while improving the accuracy of model operations.

[0133] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0134] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0135] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a device for generating an embedding vector of a digital text. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0136] like Figure 3 As shown, the digital text embedding vector generating device described in this embodiment includes:

[0137] The corpus preprocessing module 301 is used to obtain a training corpus set and preprocess the training corpus set, wherein the preprocessing includes converting digital text in the training corpus set into Chinese character strings;

[0138] A sample set construction module 302 is used to construct a feature sample set using the preprocessed training corpus, wherein the feature sample set includes numerical feature samples and arithmetic feature samples;

[0139] The first vector conversion module 303 is used to convert the numerical feature samples into word vectors to obtain an initial numerical feature embedding vector;

[0140] A second vector conversion module 304 is used to convert the arithmetic feature sample into a word vector to obtain an initial arithmetic feature embedding vector;

[0141] A joint training module 305 is used to iteratively train the preset joint neural network model using the initial numerical feature embedding vector and the initial arithmetic feature embedding vector until the model is fitted;

[0142] The embedding vector generation module 306 is used to obtain the weighted average value of the joint neural network model after model fitting to obtain the embedding vector of the digital text.

[0143] In a specific embodiment of the present application, the corpus preprocessing module 301 specifically includes:

[0144] A digital text search submodule is used to obtain a training corpus and search for digital texts in the training corpus;

[0145] A digital unit search submodule is used to obtain a preset digital unit set;

[0146] The string conversion submodule is used to convert the digital text in the training corpus into a string according to the digital unit set to obtain the numerical text corresponding to the digital text, wherein the numerical text is the Chinese string corresponding to the digital text.

[0147] In a specific embodiment of the present application, the sample set construction module 302 specifically includes:

[0148] A numerical sample set construction submodule is used to obtain numerical corpus in the training corpus and construct numerical feature samples based on the numerical corpus;

[0149] The arithmetic sample set construction submodule is used to obtain arithmetic corpus in the training corpus and construct arithmetic feature samples based on the arithmetic corpus.

[0150] In a specific embodiment of the present application, the numerical sample set construction submodule specifically includes:

[0151] A numerical text set construction unit is used to traverse the numerical corpus, obtain all numerical texts in the numerical corpus, and construct a numerical text set;

[0152] The numerical sample set construction unit is used to obtain a preset sliding window and use the sliding window to slide in the numerical text set to generate numerical feature samples.

[0153] In a specific embodiment of the present application, the first vector conversion module 303 specifically includes:

[0154] A first numerical text acquisition submodule is used to acquire all numerical texts in the numerical feature sample to obtain a first numerical text;

[0155] A first word vector conversion submodule is used to convert each Chinese character of the first numerical text into a word vector in turn to obtain a first initial embedding vector, wherein each Chinese character of the first numerical text corresponds to a first initial embedding vector;

[0156] The frequency coefficient calculation submodule is used to count the frequency coefficient of each Chinese character;

[0157] The numerical embedding vector submodule is used to construct an initial numerical feature embedding vector based on the first initial embedding vector and the frequency coefficient of each Chinese character.

[0158] In a specific embodiment of the present application, the arithmetic sample set construction submodule specifically includes:

[0159] An arithmetic text acquisition unit, used for traversing arithmetic materials and acquiring arithmetic texts in the arithmetic materials;

[0160] An operation symbol recognition unit is used to perform semantic recognition on arithmetic text to obtain the operation symbols of the arithmetic text;

[0161] A second numerical text acquisition unit, used for extracting the numerical text of the arithmetic text to obtain a second numerical text;

[0162] The arithmetic sample set construction unit is used to construct an arithmetic feature sample based on the second numerical text and the operation symbol of the arithmetic text.

[0163] In a specific embodiment of the present application, the second vector conversion module 304 specifically includes:

[0164] A second word vector conversion submodule is used to convert each Chinese character of the second numerical text into a word vector in turn to obtain a second initial embedding vector, wherein each Chinese character of the second numerical text corresponds to a second initial embedding vector;

[0165] The operator conversion submodule is used to convert the operator symbol into a word vector to obtain the operator embedding vector;

[0166] The arithmetic feature vector submodule is used to construct an initial arithmetic feature embedding vector based on the second initial embedding vector and the operator embedding vector.

[0167] In a specific embodiment of the present application, the joint neural network model includes a first neural network model and a second neural network model, and the joint training module 305 specifically includes:

[0168] A first iterative submodule, used to input the initial numerical feature embedding vector into the first neural network model, and iteratively update the first neural network model using a preset cross entropy loss function;

[0169] A second iterative submodule, used for inputting the initial arithmetic feature embedding vector into the second neural network model, and iteratively updating the second neural network model using a preset mean square error loss function;

[0170] A first gradient value calculation submodule, used to obtain the gradient value of the first neural network model after the iteration is completed, to obtain a first gradient value;

[0171] A second gradient value calculation submodule is used to obtain the gradient value of the second neural network model after the iteration is completed, so as to obtain a second gradient value;

[0172] The joint optimization submodule is used to calculate the gradient average of the first gradient value and the second gradient value, and use the gradient average to optimize the joint neural network model to obtain a fitted joint neural network model.

[0173] In this embodiment, the present application discloses a device for generating an embedding vector of a digital text, and the present application belongs to the field of artificial intelligence technology. The present application converts the character string of the digital text in the training corpus, and uses the training corpus that completes the character string conversion to construct numerical feature samples and arithmetic feature samples, converts the numerical feature samples and arithmetic feature samples into word vectors, obtains the initial numerical feature embedding vector and the initial arithmetic feature embedding vector, and uses the above two initial embedding vectors to iteratively train the joint neural network model, obtains the model weight average after model fitting, and obtains the embedding vector of the digital text. The present application converts the digital text into a Chinese character string, and then converts the Chinese character string into a word vector, retains the digital text features, and uses the arithmetic features in the training corpus to jointly train the neural network, so that the generated digital embedding vector has the ability to compare the size of numbers and perform arithmetic operations, thereby coping with more complex downstream processing tasks while improving the accuracy of model operations.

[0174] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0175] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0176] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0177] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the embedding vector generation method of digital text, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0178] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the embedding vector generation method of the digital text.

[0179] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0180] The present application discloses a computer device, and the present application belongs to the field of artificial intelligence technology. The present application converts the character string of the digital text in the training corpus, and uses the training corpus that has completed the character string conversion to construct numerical feature samples and arithmetic feature samples, converts the numerical feature samples and arithmetic feature samples into word vectors, obtains the initial numerical feature embedding vector and the initial arithmetic feature embedding vector, and uses the above two initial embedding vectors to iteratively train the joint neural network model, obtains the model weight average after model fitting, and obtains the embedding vector of the digital text. The present application converts the digital text into a Chinese character string, and then converts the Chinese character string into a word vector, retains the digital text features, and uses the arithmetic features in the training corpus to jointly train the neural network, so that the generated digital embedding vector has the ability to compare the size of numbers and perform arithmetic operations, thereby coping with more complex downstream processing tasks while improving the accuracy of model operations.

[0181] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the method for generating an embedding vector of digital text as described above.

[0182] The present application discloses a storage medium, and the present application belongs to the field of artificial intelligence technology. The present application converts the character string of the digital text in the training corpus, and uses the training corpus that has completed the character string conversion to construct numerical feature samples and arithmetic feature samples, converts the numerical feature samples and arithmetic feature samples into word vectors, obtains the initial numerical feature embedding vector and the initial arithmetic feature embedding vector, and uses the above two initial embedding vectors to iteratively train the joint neural network model, obtains the model weight average after model fitting, and obtains the embedding vector of the digital text. The present application converts the digital text into a Chinese character string, and then converts the Chinese character string into a word vector, retains the digital text features, and uses the arithmetic features in the training corpus to jointly train the neural network, so that the generated digital embedding vector has the ability to compare the size of numbers and perform arithmetic operations, thereby coping with more complex downstream processing tasks while improving the accuracy of model operations.

[0183] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0184] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0185] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for generating an embedding vector of digital text, characterized in that: include: Acquire a training corpus and preprocess the training corpus, wherein the preprocessing includes converting digital text in the training corpus into Chinese character strings; Constructing a feature sample set using the preprocessed training corpus, wherein the feature sample set includes numerical feature samples and arithmetic feature samples; Performing word vector conversion on the numerical feature sample to obtain an initial numerical feature embedding vector; Performing word vector conversion on the arithmetic feature sample to obtain an initial arithmetic feature embedding vector; Iteratively training a preset joint neural network model using the initial numerical feature embedding vector and the initial arithmetic feature embedding vector until the model is fitted; Obtaining a weighted average value of the joint neural network model after model fitting to obtain an embedding vector of the digital text; The step of obtaining a training corpus and preprocessing the training corpus specifically includes: Obtain a training corpus, and search for digital text in the training corpus; Get the preset digital unit set; Performing character string conversion on the digital text in the training corpus according to the digital unit set to obtain a numerical text corresponding to the digital text, wherein the numerical text is a Chinese character string corresponding to the digital text; The step of constructing a feature sample set using the preprocessed training corpus specifically includes: Acquire numerical corpus in the training corpus set, and construct numerical feature samples based on the numerical corpus; Acquire arithmetic corpus in the training corpus, and construct arithmetic feature samples based on the arithmetic corpus; The step of obtaining the numerical corpus in the training corpus set and constructing the numerical feature sample based on the numerical corpus specifically includes: Traversing the numerical corpus, obtaining all numerical texts in the numerical corpus, and constructing a numerical text set; Obtaining a preset sliding window, and using the sliding window to slide in the numerical text set to generate a numerical feature sample; The step of obtaining the arithmetic corpus in the training corpus and constructing the arithmetic feature sample based on the arithmetic corpus specifically includes: Traversing the arithmetic term material, and acquiring arithmetic text in the arithmetic term material; Performing semantic recognition on the arithmetic text to obtain operation symbols of the arithmetic text; Extracting the numerical text of the arithmetic text to obtain a second numerical text; An arithmetic feature sample is constructed based on the second numerical text and the operation symbol of the arithmetic text.

2. The method for generating an embedding vector of a digital text as claimed in claim 1, characterized in that: The step of converting the numerical feature sample into a word vector to obtain an initial numerical feature embedding vector specifically includes: Acquire all numerical texts in the numerical feature sample to obtain a first numerical text; Convert each Chinese character of the first numerical text into a word vector in sequence to obtain a first initial embedding vector, wherein each Chinese character of the first numerical text corresponds to a first initial embedding vector; Counting the frequency coefficient of each of the Chinese characters; The initial numerical feature embedding vector is constructed based on the first initial embedding vector and the frequency coefficient of each of the Chinese characters.

3. The method for generating an embedding vector of a digital text as claimed in claim 1, characterized in that: The step of converting the arithmetic feature sample into a word vector to obtain an initial arithmetic feature embedding vector specifically includes: Convert each Chinese character of the second numerical text into a word vector in sequence to obtain a second initial embedding vector, wherein each Chinese character of the second numerical text corresponds to a second initial embedding vector; Convert the operator symbol into a word vector to obtain an operator embedding vector; The initial arithmetic feature embedding vector is constructed based on the second initial embedding vector and the operator embedding vector.

4. The method for generating an embedding vector of a digital text according to any one of claims 1 to 3, characterized in that: The joint neural network model includes a first neural network model and a second neural network model, and the step of iteratively training the preset joint neural network model using the initial numerical feature embedding vector and the initial arithmetic feature embedding vector until the model is fitted specifically includes: Inputting the initial numerical feature embedding vector into the first neural network model, and iteratively updating the first neural network model using a preset cross entropy loss function; Inputting the initial arithmetic feature embedding vector into the second neural network model, and iteratively updating the second neural network model using a preset mean square error loss function; Obtaining a gradient value of the first neural network model after iteration to obtain a first gradient value; Obtaining a gradient value of the second neural network model after iteration to obtain a second gradient value; The gradient average of the first gradient value and the second gradient value is calculated, and the joint neural network model is optimized using the gradient average to obtain the fitted joint neural network model.

5. A digital text embedding vector generation device, characterized in that: The digital text embedding vector generating device performs the steps of the digital text embedding vector generating method according to any one of claims 1 to 4, and the digital text embedding vector generating device comprises: A corpus preprocessing module, used to obtain a training corpus set and preprocess the training corpus set, wherein the preprocessing includes converting digital text in the training corpus set into Chinese character strings; A sample set construction module, used to construct a feature sample set using the preprocessed training corpus, wherein the feature sample set includes numerical feature samples and arithmetic feature samples; A first vector conversion module, used to perform word vector conversion on the numerical feature sample to obtain an initial numerical feature embedding vector; A second vector conversion module is used to perform word vector conversion on the arithmetic feature sample to obtain an initial arithmetic feature embedding vector; A joint training module, used for iteratively training a preset joint neural network model using the initial numerical feature embedding vector and the initial arithmetic feature embedding vector until the model is fitted; The embedding vector generation module is used to obtain the weighted average value of the joint neural network model after model fitting and the embedding vector of the digital text.

6. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for generating an embedding vector of a digital text as described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for generating an embedding vector for digital text as described in any one of claims 1 to 4.

Images