Text information processing model training method and apparatus, and storage medium

Abstract

This invention provides a text information processing model training method, text information processing method, device, electronic device, software program, and storage medium. Applicable scenarios include, but are not limited to, maps, navigation, autonomous driving, vehicle-to-everything (V2X) communication, and vehicle-road cooperative environments. The method includes: performing probabilistic prediction processing on text information to obtain text embedding feature vectors and predicted probability distribution information; performing probabilistic prediction processing on label information using a deep neural network of the text processing model to obtain label embedding feature vectors; performing one-hot encoding processing on the label information using a one-hot encoding network of the text processing model to obtain one-hot encoded feature vectors; calculating actual probability distribution information; and adjusting the loss function of the text processing model based on the predicted and actual probability distribution information. This saves training time for the text processing model and improves its classification accuracy.

Description

Technical Field

[0001] This invention relates to text information processing technology, and more particularly to text information processing model training methods, text information processing methods, devices, electronic devices, software programs, and storage media. Background Technology

[0002] In text information processing, due to the wide range of text content domains, the text information classification technology mainly uses Long Short-Term Memory (LSTM) networks. However, if the text information is long, this method will lose a lot of key information, resulting in poor classification results. Another commonly used technique is to use Convolutional Neural Networks (CNN). When using CNN, due to its window feature, it can extract features with different spans. This approach has good parallelism and the model is easy to train, but it cannot grasp the relationship between words or positional features, which also affects the accuracy of text information classification. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a text information processing model training method, a text information processing method, an apparatus, an electronic device, a software program, and a storage medium, which can adjust the loss function of the text processing model by predicting probability distribution information and actual probability distribution information, save the training time of the text processing model, improve the classification accuracy of the text processing model, and enhance the user experience.

[0004] The technical solution of this invention is implemented as follows:

[0005] This invention provides a method for training a text information processing model, comprising:

[0006] In response to a text information processing request, retrieve text information and tag information;

[0007] The text information is subjected to probability prediction processing through the text prediction network of the text processing model to obtain the text embedding feature vector and prediction probability distribution information of the text information.

[0008] The tag information is embedded using the deep neural network of the text processing model to obtain the tag embedding feature vector of the tag information.

[0009] The tag information is processed by one-hot encoding through the one-hot encoding network of the text processing model to obtain the one-hot encoded feature vector of the tag information.

[0010] Based on the text embedding feature vector, the tag embedding feature vector, and the one-hot encoding feature vector, calculate the actual probability distribution information;

[0011] Based on the predicted probability distribution information and the actual probability distribution information, the loss function of the text processing model is adjusted to obtain the classification result of the text information to be processed through the text processing model.

[0012] This invention also provides a text information processing method, including:

[0013] Obtain the text information to be processed;

[0014] The encoder of the text processing model determines at least one word-level latent variable corresponding to the text information to be processed.

[0015] The decoder of the text processing model generates a text classification result corresponding to the word-level latent variable and the probability of the text classification result being selected, based on the at least one word-level latent variable.

[0016] Based on the selection probability of the text classification result, at least one text classification result is selected.

[0017] This invention also provides a text information processing model training device, comprising:

[0018] The information transmission module is used to respond to text information processing requests and obtain text information and tag information;

[0019] The information processing module is used to perform probability prediction processing on the text information through the text prediction network of the text processing model to obtain the text embedding feature vector and prediction probability distribution information of the text information.

[0020] The information processing module is used to embed the tag information through the deep neural network of the text processing model to obtain the tag embedding feature vector of the tag information.

[0021] The information processing module is used to perform one-hot encoding processing on the tag information through the one-hot encoding network of the text processing model to obtain the one-hot encoded feature vector of the tag information.

[0022] The information processing module is used to calculate the actual probability distribution information based on the text embedding feature vector, the tag embedding feature vector, and the one-hot encoding feature vector;

[0023] The information processing module is used to adjust the loss function of the text processing model based on the predicted probability distribution information and the actual probability distribution information, so as to obtain the classification result of the text information to be processed through the text processing model.

[0024] In the above scheme,

[0025] The information processing module is used to determine the type and quantity of the tag information;

[0026] The information processing module is used to determine the number of dimensions of the tag embedding feature vector based on the index information of the tag embedding feature vector.

[0027] The information processing module is used to calculate the tag information matrix of the tag information based on the number of types and the number of dimensions of the tag information;

[0028] The information processing module is used to perform embedding processing on the label information matrix through the embedding layer network of the deep neural network to obtain the label embedding feature vector of the label information.

[0029] In the above scheme,

[0030] The information processing module is used to calculate the product of the text embedding feature vector and the tag embedding feature vector;

[0031] The information processing module is used to perform nonlinear transformation processing on the product of the text embedding feature vector and the tag embedding feature vector to obtain the similarity of the text embedding feature vectors;

[0032] The information processing module is used to perform vector smoothing on the one-hot encoded feature vector based on the similarity of the text embedding feature vectors to obtain the actual probability distribution information.

[0033] In the above scheme,

[0034] The information processing module is used to parse the text information processing request and determine the target object included in the text information processing request and the financial scenario corresponding to the target object;

[0035] The information processing module is used to determine the historical behavior parameters of the target object and the historical parameters of the financial scenario in the financial scenario.

[0036] The information processing module is used to perform cross-filtering of the historical behavior parameters of the target object and the historical parameters of the financial scenario based on the target object, and to obtain text information that matches the target object.

[0037] The information processing module is used to perform cross-filtering of the historical behavior parameters of the target object and the historical parameters of the financial scenario based on the target object, and to obtain tag information that matches the target object.

[0038] In the above scheme,

[0039] The information processing module is used to calculate the relative entropy of the text processing model based on the predicted probability distribution information and the actual probability distribution information.

[0040] The information processing module is used to adjust the loss function of the text processing model by replacing the cross entropy of the text processing model with the relative entropy of the text processing model.

[0041] This invention also provides a text information processing device, comprising:

[0042] The data transmission module is used to acquire the text information to be processed.

[0043] The data processing module determines at least one word-level latent variable corresponding to the text information to be processed through the encoder of the text processing model.

[0044] The data processing module, through the decoder of the text processing model, generates a text classification result corresponding to the word-level latent variable and the selection probability of the text classification result based on the at least one word-level latent variable.

[0045] The data processing module selects at least one text classification result based on the selection probability of the text classification result.

[0046] This invention also provides an electronic device, the electronic device comprising:

[0047] Memory, used to store executable instructions;

[0048] The processor, when running the executable instructions stored in the memory, implements the aforementioned text information processing model training method.

[0049] The present invention discloses a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the aforementioned text information processing model training method.

[0050] The embodiments of the present invention have the following beneficial effects:

[0051] This invention responds to a text information processing request by acquiring text information and tag information; using a text prediction network of a text processing model, it performs probability prediction processing on the text information to obtain a text embedding feature vector and predicted probability distribution information; using a deep neural network of the text processing model, it performs embedding processing on the tag information to obtain a tag embedding feature vector; using a one-hot encoding network of the text processing model, it performs one-hot encoding processing on the tag information to obtain a one-hot encoded feature vector; based on the text embedding feature vector, the tag embedding feature vector, and the one-hot encoded feature vector, it calculates the actual probability distribution information; based on the predicted probability distribution information and the actual probability distribution information, it adjusts the loss function of the text processing model to achieve a classification result for the text information to be processed. This allows for adjusting the loss function of the text processing model by using predicted and actual probability distribution information, saving training time, improving classification accuracy, and enhancing the user experience. Attached Figure Description

[0052] Figure 1 This is a schematic diagram illustrating the usage environment of the text information processing model training method provided in this embodiment of the invention;

[0053] Figure 2 This is a schematic diagram of the composition structure of the text information processing model training device provided in an embodiment of the present invention;

[0054] Figure 3 This is an optional flowchart illustrating the text information processing model training method provided in an embodiment of the present invention.

[0055] Figure 4 This is a schematic diagram of the model structure of the text processing model in an embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of an optional structure of the text prediction network in an embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of an optional translation process of the text prediction network in an embodiment of the present invention;

[0058] Figure 7 This is a schematic diagram of an optional encoder structure in the text prediction network of this invention.

[0059] Figure 8 This is a schematic diagram of vector concatenation in the encoder of the text prediction network in an embodiment of the present invention;

[0060] Figure 9This is a schematic diagram of the encoding process of the encoder in the text prediction network in an embodiment of the present invention;

[0061] Figure 10 This is a schematic diagram of the decoding process of the decoder in the text prediction network in an embodiment of the present invention;

[0062] Figure 11 This is a schematic diagram of the decoding process of the decoder in the text prediction network in an embodiment of the present invention;

[0063] Figure 12 This is a schematic diagram illustrating the processing procedure of a deep neural network according to an embodiment of the present invention;

[0064] Figure 13 This is a schematic diagram of the one-hot coding network in an embodiment of the present invention;

[0065] Figure 14 This is an optional flowchart illustrating the text information processing method provided in an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments, but it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0068] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0069] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0070] 1) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0071] 2) Neural Network (NN): Artificial Neural Network (ANN), also known as neural network or neural network-like network, is a mathematical or computational model in the fields of machine learning and cognitive science that imitates the structure and function of biological neural networks (the central nervous system of animals, especially the brain) and is used to estimate or approximate functions.

[0072] 3) Transformers: A new network structure that uses an attention mechanism instead of the traditional encoder-decoder model that must rely on other neural networks.

[0073] 4) Machine Translation (MT): This falls under the category of computational linguistics, and its research focuses on using computer programs to translate text or speech from one natural language into another. Neural Machine Translation (NMT) is a technique that uses neural networks for machine translation.

[0074] 5) Encoder-decoder structure: A commonly used network structure in machine translation technology. It consists of two parts: an encoder and a decoder. The encoder converts the input text into a series of context vectors that can express the features of the input text. The decoder receives the output of the encoder as its own input and outputs the corresponding text sequence in another language.

[0075] 6) Bidirectional attention neural network (BERT Bidirectional Encoder Representations from Transformers) is a bidirectional attention neural network model.

[0076] 7) Token: Before any actual processing is performed on the input text, it needs to be divided into language units such as words, punctuation marks, numbers, or alphanumeric characters. These units are called tokens.

[0077] 8) Softmax: The normalized exponential function, a generalization of the logistic function. It can "compress" a K-dimensional vector containing arbitrary real numbers into another K-dimensional real vector, such that each element is in the range [0,1], and the sum of all elements is 1.

[0078] 10) Text classification: Classify text into a target category within a category system.

[0079] 11) Word segmentation: Using Chinese word segmentation tools to segment Chinese text, resulting in a set of fine-grained words. Stop words: Characters or words that contribute nothing to the semantics of the text or whose contribution is negligible. Cosine similarity: The cosine similarity between two texts represented as vectors.

[0080] Figure 1 This is a schematic diagram illustrating a usage scenario of the text information processing model training method provided in this embodiment of the invention. (See attached diagram.) Figure 1 The terminals (including terminals 10-1 and 10-2) are equipped with software clients capable of displaying corresponding financial text information. These clients or plugins may be used for financial activities involving virtual or physical resources, or for payments made using virtual resources (such as QQ coins). The target audience can obtain and view the financial text information through the corresponding client, and trigger corresponding text information processing processes during the text information processing (e.g., a payment mini-program in instant messaging software or a process in instant messaging software for purchasing funds or stocks; by processing the text information, the target audience's emotional state towards different funds or stocks can be obtained to determine their preference for different funds or stocks; or, by processing the text information, the time the target audience spends on various types of securities in the information they browse can be obtained to determine their preference for different funds or stocks). The terminals connect to server 200 via network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both, using a wireless link to achieve data transmission.

[0081] As an example, server 200 is used to deploy the text information processing model training device to implement the text information processing model training method provided by the present invention, so as to obtain text information and tag information in response to text information processing requests; perform probability prediction processing on the text information through the text prediction network of the text processing model to obtain the text embedding feature vector and prediction probability distribution information of the text information; perform embedding processing on the tag information through the deep neural network of the text processing model to obtain the tag embedding feature vector of the tag information; and perform one-hot encoding processing on the tag information through the one-hot encoding network of the text processing model to obtain... The text processing model obtains a one-hot encoded feature vector of the tag information; calculates the actual probability distribution information based on the text embedding feature vector, the tag embedding feature vector, and the one-hot encoded feature vector; adjusts the loss function of the text processing model based on the predicted probability distribution information and the actual probability distribution information to obtain the classification result of the text information to be processed through the text processing model, and determines the sentiment state of the text information to be processed through the classification result, and performs related operations based on the sentiment state of the text information, such as the purchase behavior of securities such as funds or stocks, and the sentiment recognition of user dialogues, etc. The specific content of the behavior is not limited in this application.

[0082] Of course, the text information processing model training device provided by this invention can be applied to environments where financial activities are conducted using virtual or physical resources, or where payments are made using physical financial resources (including but not limited to various types of physical financial resource change environments), or where information is exchanged using social software. In various types of physical financial resource financial activities or payments made using virtual resources, financial text information from different data sources is typically processed, and finally presented on the user interface (UI) along with the corresponding financial text information corresponding to the target object selected by the target object. The emotional state of the text information to be processed formed by the target object in the current display interface (e.g., the emotional state regarding real-time price fluctuations of securities such as funds or stocks, or the emotional state regarding the rise and fall of futures) can also be invoked by other applications. It should be noted that the emotional state involved in this application can include at least the following: positive (positive) emotion, negative (negative) emotion, and neutral emotion. The emotional state can reflect the target object's viewpoint and emotional information, and has broad application prospects in areas such as topic discovery, targeted advertising, and after-sales service evaluation.

[0083] The text information processing model training method provided in this application is based on artificial intelligence (AI). AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making functions.

[0084] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0085] In the embodiments of this application, the main artificial intelligence software technologies involved include the aforementioned speech processing technologies and machine learning. For example, it may involve Automatic Speech Recognition (ASR) technology in speech technology, including speech signal preprocessing, speech signal frequency analyzing, speech signal feature extraction, speech signal feature matching / recognition, and speech training.

[0086] For example, this could involve machine learning (ML), a multidisciplinary field encompassing probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning typically includes techniques such as deep learning, which includes artificial neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and deep neural networks (DNNs).

[0087] The structure of the text information processing model training device according to an embodiment of the present invention will be described in detail below. The text information processing model training device can be implemented in various forms, such as a dedicated terminal with text information processing model training device processing function, or a server equipped with text information processing model training device processing function, for example, the preceding... Figure 1 Server 200 in the middle. Figure 2 This is a schematic diagram of the composition structure of the text information processing model training device provided in an embodiment of the present invention. It can be understood that... Figure 2 This only shows an exemplary structure of the text information processing model training device, not the entire structure; it can be implemented as needed. Figure 2 The structure shown may be part or all of the structure.

[0088] The text information processing model training device provided in this embodiment of the invention includes: at least one processor 201, a memory 202, a user interface 203, and at least one network interface 204. The various components in the text information processing model training device are coupled together through a bus system 205. It can be understood that the bus system 205 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 205 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 205.

[0089] The user interface 203 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0090] It is understood that memory 202 can be volatile memory or non-volatile memory, or both. In this embodiment of the invention, memory 202 is capable of storing data to support the operation of the terminal (e.g., 10-1). Examples of this data include any computer programs used to operate on the terminal (e.g., 10-1), such as operating systems and applications. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications.

[0091] In some embodiments, the text information processing model training device provided in this invention can be implemented using a combination of hardware and software. For example, the text information processing model training device provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the text information processing model training method provided in this invention. For instance, the processor in the form of a hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0092] As an example of the text information processing model training device provided in this embodiment of the invention, which adopts a combination of hardware and software, the text information processing model training device provided in this embodiment of the invention can be directly embodied as a combination of software modules executed by processor 201. The software modules can be located in a storage medium, which is located in memory 202. Processor 201 reads the executable instructions included in the software modules in memory 202 and combines them with necessary hardware (e.g., including processor 201 and other components connected to bus 205) to complete the text information processing model training method provided in this embodiment of the invention.

[0093] As an example, processor 201 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0094] As an example of the hardware implementation of the text information processing model training device provided in this embodiment of the invention, the device provided in this embodiment of the invention can be directly executed by a processor 201 in the form of a hardware decoding processor. For example, it can be executed by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the text information processing model training method provided in this embodiment of the invention.

[0095] In this embodiment of the invention, the memory 202 is used to store various types of data to support the operation of the text information processing model training device. Examples of such data include: any executable instructions for operation on the text information processing model training device, such as executable instructions that can be included in the executable instructions, implementing the text information processing model training method of this embodiment of the invention.

[0096] In other embodiments, the text information processing model training device provided in this invention can be implemented in software. Figure 2 A text information processing model training device stored in memory 202 is shown. This device can be software in the form of programs and plugins, and includes a series of modules. As an example of a program stored in memory 202, it may include the text information processing model training device. The text information processing model training device includes the following software modules: an information transmission module 2081 and an information processing module 2082. When the software modules in the text information processing model training device are read into RAM and executed by processor 201, the text information processing model training method provided in this embodiment of the invention will be implemented. The functions of each software module in the text information processing model training device include:

[0097] The information transmission module is used to respond to text information processing requests and obtain text information and tag information;

[0098] The information processing module is used to perform probability prediction processing on the text information through the text prediction network of the text processing model to obtain the text embedding feature vector and prediction probability distribution information of the text information.

[0099] The information processing module is used to embed the tag information through the deep neural network of the text processing model to obtain the tag embedding feature vector of the tag information.

[0100] The information processing module is used to perform one-hot encoding processing on the tag information through the one-hot encoding network of the text processing model to obtain the one-hot encoded feature vector of the tag information.

[0101] The information processing module is used to calculate the actual probability distribution information based on the text embedding feature vector, the tag embedding feature vector, and the one-hot encoding feature vector;

[0102] The information processing module is used to adjust the loss function of the text processing model based on the predicted probability distribution information and the actual probability distribution information, so as to obtain the classification result of the text information to be processed through the text processing model.

[0103] After the text processing model is trained, the electronic device can continue to deploy a text information processing device. The data transmission module acquires the text information to be processed. The data processing module, through the encoder of the text processing model, determines at least one word-level latent variable corresponding to the text information to be processed. The data processing module, through the decoder of the text processing model, generates a text classification result corresponding to the at least one word-level latent variable and the selection probability of the text classification result based on the at least one word-level latent variable. The data processing module selects at least one text classification result based on the selection probability of the text classification result.

[0104] according to Figure 2 The electronic device shown, in one aspect of this application, also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the above-described text information processing model training method.

[0105] Continue to combine Figure 1The illustrated use case explains the text information processing model training method provided in this application. In this method, terminals (including terminals 10-1 and 10-2) obtain market data on the allocation process of financial resources such as bonds, funds, or stocks from the corresponding server 200 via network 300 and comment on it. These comments can be used as text information to be processed, for example, comments on market data through platforms such as WeChat Pay, fund or stock trading apps. Due to the wide range of text content domains, the text information classification technology used is mainly based on Long Short-Term Memory (LSTM) networks. However, if the text information is long, this method loses a large amount of key information, resulting in poor classification performance. Another commonly used technique is to use Convolutional Neural Networks (CNNs). CNNs, due to their window features, extract features with different spans. This approach has good parallelism and the model is easy to train, but it cannot grasp the relationships between words or positional features, which also affects the analysis of the sentiment state of the text information. Especially in the financial field, its adaptability is poor, affecting the user experience.

[0106] If we use text processing models to categorize financial information read by users and determine user preferences for different financial products based on browsing time, identifying the text's category is a crucial step in text category annotation. Traditional methods for determining text categories involve manually labeling several texts to obtain training samples. These training samples are then used to train a machine learning model, such as a neural network, to create a mapping model. The text to be processed is then input into this mapping model, which determines its category. However, the manual labeling process for obtaining training samples is extremely labor-intensive. Furthermore, since the mapping model is trained on these manually labeled samples, the quality of the determined category is heavily dependent on the quality of the manual labeling.

[0107] Therefore, in the context of browsing financial information, neither emotional state classification nor information type classification can achieve accurate classification. At the same time, text processing models rely on long-term training of samples, resulting in excessively high training costs.

[0108] To address the aforementioned deficiencies, refer to Figure 3 , Figure 3 This is an optional flowchart illustrating the text information processing model training method provided in this embodiment of the invention. The target object can be selected for use in different financial scenarios. It is understood that... Figure 3The steps shown can be performed by various electronic devices that run text information processing model training devices, such as dedicated terminals with text information processing functions (securities machines or mobile phones, such as fund or stock machines), electronic devices, or financial mini-programs. The following section focuses on... Figure 3 The steps shown are explained.

[0109] Step 301: The text information processing model training device responds to the text information processing request and obtains text information and label information.

[0110] In some embodiments of the present invention, taking a financial text information processing scenario as an example, the financial text information to be processed can be obtained in the following ways:

[0111] The text information processing request is parsed to determine the target object included in the text information processing request and the financial scenario corresponding to the target object; in the financial scenario, the historical behavior parameters of the target object and the historical parameters of the financial scenario are determined; by performing data cross-filtering processing on the historical behavior parameters of the target object and the historical parameters of the financial scenario, text information matching the target object is obtained.

[0112] When the target audience is a user participating in financial transactions, the user's historical click behavior during online financial transactions and the duration of browsing corresponding financial product advertisements are recorded through a subscription service and stored in a Redis Remote Dictionary Server. The online recommendation system retrieves the user's historical click behavior and the duration of browsing financial product advertisements when the user's instant messaging client is accessed, thereby determining the target audience's historical behavioral parameters. Simultaneously, the historical parameters of the financial scenario include the historical returns of financial products and changes in trading volume. When cross-filtering the historical behavioral parameters of the target audience and the historical parameters of the financial scenario, financial product A is identified through the historical behavioral parameters of the target audience. Then, querying the historical returns and trading volume changes of A yields a text message matching the target audience: "On XX year XX month XX day, the return of financial product A was XX, and the trading volume was XX." Additionally, the system also retrieves the matching tag information for the target audience: "Financial Product A," "Return," and "Trading Volume."

[0113] In some embodiments of the present invention, when recommending funds or stocks through securities trading software, the user sends a text information processing request to the financial server through the client of the securities trading software. The text information processing request includes target object identification parameters and scenario parameters. After parsing the text information processing request to obtain the target object identification parameters and scenario parameters, the target object and its corresponding data interface can be determined based on the mapping relationship of the target object identification parameters. Through the data interface, the historical behavior parameters of the target object can be retrieved from the database of the financial service provider. For example, the target object may have added a fund or stock to its favorites, commented on a fund or stock, or purchased a fund or stock. Through the mapping relationship of the scenario parameters, the scenario corresponding to the target object can be determined as a financial scenario, and the data interface for the scenario parameters can be determined by... Given the diverse data types and the involvement of target object privacy data in financial scenarios, each scenario parameter's data interface corresponds to only one fixed financial server. Utilizing this interface, the financial server retrieves historical parameters from a historical information database. These parameters could include market trend data for funds or stocks, or trading volume changes for a specific sector of funds or stocks. By cross-filtering the target object's historical behavior parameters within the historical parameters of the financial scenario, the system identifies the target object's operational information at different times within the financial scenario, which is then used as matching text information. For example, cross-filtering could yield information such as "which fund or stock did the target object purchase as market trend data changed," or "what comments did the target object make on a particular fund or stock as trading volume changed in a specific sector of funds or stocks."

[0114] Step 302: The text information processing model training device performs probability prediction processing on the text information through the text prediction network of the text processing model to obtain the text embedding feature vector and prediction probability distribution information of the text information.

[0115] refer to Figure 4 , Figure 4This is a schematic diagram of the model structure of the text processing model in this embodiment of the invention. The text processing model includes: a text prediction network, a deep neural network, and a one-hot encoding network. The text prediction network performs probabilistic prediction processing on the text information to obtain the text embedding feature vector and prediction probability distribution information. The deep neural network performs embedding processing on the tag information to obtain the tag embedding feature vector. The one-hot encoding network performs one-hot encoding processing on the tag information to obtain the one-hot encoded feature vector. The working processes of the text prediction network, the deep neural network, and the one-hot encoding network are described below.

[0116] In some embodiments of the present invention, the text prediction network can be a bidirectional attention neural network model (BERTBidirectional Encoder Representations from Transformers). Continue to refer to Figure 5 , Figure 5 This is a schematic diagram of an optional structure of the text processing model in an embodiment of the present invention. The Encoder comprises N = 6 identical layers, each containing two sub-layers. The first sub-layer is a multi-head attention layer, followed by a simple fully connected layer. Each sub-layer includes residual connections and normalization.

[0117] The decoder consists of N=6 identical layers, where each layer is different from the encoder. Each layer contains three sub-layers: a self-attention layer, an encoder-decoder attention layer, and a fully connected layer. The first two sub-layers are based on a multi-head attention layer.

[0118] Continue to refer to Figure 6 , Figure 6This is a schematic diagram of an optional word-level category classification process of the text processing model in this embodiment of the invention. The encoder and decoder parts each contain six encoders and six decoders. The inputs entering the first encoder combine embedding and positional embedding. After passing through the six encoders, the output is sent to each decoder in the decoder part. The input target is "I am a student t". After processing by the text processing model, the output text category classification result is "student".

[0119] Continue to refer to Figure 7 , Figure 7 This is a schematic diagram of an optional structure of the encoder in the text processing model of the present invention. Its input consists of a query (Q) and a key (K) of dimension d and a value (V) of dimension d. The dot product of the query is calculated for all keys, and the softmax function is applied to obtain the weight of the value.

[0120] Continue to refer to Figure 7 , Figure 7 This embodiment of the invention illustrates the encoder vector diagram in the text processing model. Q, K, and V are obtained by multiplying the input encoder vector x by W^Q, W^K, and W^V. W^Q, W^K, and W^V have dimensions of (512, 64) in the text. Assuming the input vector has dimensions of (m, 512), where m represents the number of characters, the dimensions of Q, K, and V obtained after multiplying the input vector by W^Q, W^K, and W^V are (m, 64).

[0121] Continue to refer to Figure 8 , Figure 8 This is a schematic diagram of vector concatenation in the encoder of the text processing model in this embodiment of the invention. Z0 to Z7 are the corresponding 8 parallel heads (with dimensions of (m, 64)). Concatenating these 8 heads yields a dimension of (m, 512). Finally, multiplying by W^O results in an output matrix with a dimension of (m, 512), which then matches the dimension of the next encoder.

[0122] Continue to refer to Figure 9 , Figure 9This is a schematic diagram of the encoding process of the encoder in the text processing model of this invention. x1 reaches the state z1 after self-attention. The tensor that has passed self-attention needs to be processed by a residual network and LaterNorm, and then enters a fully connected feedforward network. The feedforward network performs the same operations, including residual processing and normalization. Finally, the output tensor can enter the next encoder. This process is iterated six times, and the result of the iterations enters the decoder.

[0123] Continue to refer to Figure 10 , Figure 10 This is a schematic diagram of the decoding process of the decoder in the text processing model of this invention, wherein the decoder's input and output and the decoding process are as follows:

[0124] Output: The probability distribution of the output word at position i;

[0125] Input: the output of the encoder & the output of the decoder at position i-1. Therefore, the attention in the middle is not self-attention; its K and V come from the encoder, and Q comes from the output of the decoder at the previous position.

[0126] Continue to refer to Figure 11 , Figure 11 This is a schematic diagram of the decoding process of the decoder in the text prediction network in this embodiment of the invention. The vector output by the last decoder in the decoder network passes through a Linear layer and a softmax layer. The function of the Linear layer is to map the vector output from the decoder into a logits vector. Then, the softmax layer converts this logits vector into a probability value, and finally finds the position of the maximum probability, thus completing the decoder output.

[0127] Ultimately, as Figure 4 As shown, the text prediction network uses BERT encoding, and then uses the [CLS] encoding as the encoding of the entire text information, which is then input into the softmax classifier. The output probability distribution information of the predicted class is denoted as: y perd .

[0128] Step 303: The text information processing model training device performs embedding processing on the tag information through the deep neural network of the text processing model to obtain the tag embedding feature vector of the tag information.

[0129] refer to Figure 12 , Figure 12This is a schematic diagram of the deep neural network processing procedure according to an embodiment of the present invention. In some embodiments of the present invention, the tag information is embedded using the deep neural network of the text processing model to obtain the tag embedding feature vector of the tag information. This can be achieved in the following ways:

[0130] The number of types of the tag information is determined; based on the index information of the tag embedding feature vector, the number of dimensions of the tag embedding feature vector is determined; according to the number of types and the number of dimensions of the tag information, the tag information matrix of the tag information is calculated. For example, if the number of types of tag information is N and the dimension of each tag embedding feature vector is d, then the tag information matrix of the tag information is N*d. The tag information matrix is ​​embedded through the embedding layer network of the deep neural network to obtain the tag embedding feature vector of the tag information. The tag embedding feature vector can map sparse category features to a low-dimensional space that preserves semantic relationships. The distance between vectors represents the distance between categories, making the category labels no longer mutually independent and equidistant. Combining with the previous embodiment, the text information matching the target object is "The return of financial product A on XX year XX month XX day was XX, and the trading volume was XX". The tag information matching the target object is "financial product A", "return", and "trading volume". In this case, the number of types of tag information is 3, and the dimension of each tag embedding feature vector is 1. Therefore, the tag information matrix of the tag information is 3*1.

[0131] Step 304: The text information processing model training device performs one-hot encoding processing on the tag information through the one-hot encoding network of the text processing model to obtain the one-hot encoded feature vector of the tag information.

[0132] Step 305: The text information processing model training device calculates the actual probability distribution information based on the text embedding feature vector, the label embedding feature vector, and the one-hot encoding feature vector.

[0133] This can be achieved by calculating the product of the text embedding feature vector and the tag embedding feature vector. A non-linear transformation is then applied to this product to obtain the similarity of the text embedding feature vectors. Based on this similarity, the one-hot encoded feature vector is smoothed to obtain the actual probability distribution information. (Referencing...) Figure 13 , Figure 13This is a schematic diagram of the one-hot encoding network in an embodiment of the present invention. Each label information can be encoded in one-hot encoding format. The input layer has a size of 1*N, where N is the number of label information. The hidden layer can be represented as a weight matrix W of size V*N, where each row represents the embedding feature vector of a label information ID, where V is the length of the label information, N is the number of neurons in the hidden layer, and the dimension of the embedding feature vector is also included. The weight matrix from the hidden layer to the output layer can be represented by O, with a size of N*V. The output from the input layer to the hidden layer can be expressed as Equation 1:

[0134]

[0135] Where X is the feature vector of the label information input to the input layer, T is the working time of the one-hot encoding network, and h is the output of the hidden layer.

[0136] The output from the hidden layer to the output layer can be expressed as Equation 2:

[0137] u=O T Formula 2

[0138] The softmax function is used to normalize the output to [0, 1].

[0139] In this process, because different texts of the same category have different actual label biases, it is necessary to calculate the actual probability distribution information. For example, the text "Shanghai XX Property Management Co., Ltd.'s main revenue sources: property fees, parking fees, water and electricity fees" belonging to the "utility fees" category generally biases towards the "utility fees" category, while the text "Shanghai Jiaoxing Property Management Co., Ltd.'s main revenue sources: property fees, parking fees, water and electricity fees, and also owns xxxx apartment hotels" contains texts in the "utility fees" category as well as texts in the "hotels and inns" category. Although both texts belong to the "utility fees" category, their category probability distributions are actually different. Therefore, after calculating the label embedding feature vector, it is also necessary to calculate the similarity between the label embedding feature vector and the text embedding feature vector to obtain a more accurate category probability distribution. As shown in the above process, the label embedding feature vector and the text embedding feature vector can be multiplied and nonlinearly transformed by an activation function (such as ReLU, than, etc., which are not specifically limited in this application), that is: sim = f(label emb *text embFinally, the `sim` function is used to smooth the One-Hot vectors of the tags, so that the final probability distribution is no longer a 0 / 1 pattern, but a probability distribution that represents both the tag category distance and the text relevance. Therefore, the `sim` function and the tag One-Hot vectors are concatenated, and then mapped to the probability space using the softmax function to obtain the final actual probability distribution information, denoted as y. true .

[0140] Step 306: The text information processing model training device adjusts the loss function of the text processing model based on the predicted probability distribution information and the actual probability distribution information, so as to obtain the classification result of the text information to be processed through the text processing model.

[0141] In some embodiments of the present invention, the loss function of the text processing model is adjusted based on the predicted probability distribution information and the actual probability distribution information, which can be achieved in the following ways:

[0142] Based on the predicted probability distribution information and the actual probability distribution information, the relative entropy of the text processing model is calculated. The relative entropy of the text processing model is used to replace the cross entropy of the text processing model to adjust its loss function. Traditional text classification uses cross-entropy to calculate the loss function by comparing the predicted probability distribution with the One-Hot encoding. However, this application adjusts the loss function of the text processing model by calculating not only the difference in probabilities but also the distance between the two probability distributions. Therefore, this application uses relative entropy (KL divergence) to measure the distance between the actual probability distribution and the predicted probability distribution, as shown in Formula 3.

[0143]

[0144] Where N represents the number of categories of tag information, Represents the actual probability distribution. Denotes the predicted probability distribution, KL divergence () represents the relative entropy function.

[0145] After training is complete, the text processing model can be deployed on the corresponding financial information processing server, as shown in the reference. Figure 14 , Figure 14 An optional flowchart of the text information processing method provided in this embodiment of the invention specifically includes the following steps:

[0146] Step 1401: Obtain the text information to be processed.

[0147] Step 1402: Determine at least one word-level latent variable corresponding to the text information to be processed through the encoder of the text processing model.

[0148] Step 1403: Using the decoder of the text processing model, generate a text classification result corresponding to the word-level latent variable and the selection probability of the text classification result based on the at least one word-level latent variable.

[0149] Step 1404: Sort the selection probabilities of the text classification results, and select at least one text classification result according to the sorting results of the selection probabilities.

[0150] For example, given the text information 1) "Shanghai XX Property Management Co., Ltd.'s main revenue sources are: property management fees, parking fees, and water and electricity fees," and the text information 2) "Shanghai Jiaoxing Property Management Co., Ltd.'s main revenue sources are: property management fees, parking fees, and water and electricity fees, and it also owns xxxx apartment hotels," the text processing model provided in this application processes the text information 1), and the probability of the text classification result being selected is: "Life Payments" 0.8, "Revenue Expenses" 0.7. Therefore, the text classification result can be determined as "Life Payments."

[0151] For text information 2), after processing by the text processing model provided in this application, the probability of the selected text classification result is: "Life payment" 0.7, "Revenue expenses" 0.9. Therefore, the text classification result can be determined as "Revenue expenses", thereby achieving accurate classification of similar text information.

[0152] In some embodiments of the present invention, in a financial information processing scenario, the text information processing request can be parsed to determine the target object included in the text information processing request and the financial scenario corresponding to the target object; in the financial scenario, the historical behavior parameters of the target object and the historical parameters of the financial scenario are determined; based on the target object, the historical behavior parameters of the target object and the historical parameters of the financial scenario are subjected to data cross-filtering processing to obtain text information matching the target object; based on the target object, the historical behavior parameters of the target object and the historical parameters of the financial scenario are subjected to data cross-filtering processing to obtain tag information matching the target object. Wherein, the target object is a user participating in the trading of securities such as funds or stocks; the historical parameters of the financial scenario can be the trading information of any fund or stock listed on a stock exchange, or individual stock data corresponding to a stock and market data corresponding to the overall market; the historical behavior parameters of the target object can be text records of the user's operations on various financial products, such as funds or stocks, securities, futures, and bonds, as well as historical evaluations of the aforementioned financial products, or the duration of the user's browsing of various financial products, such as funds or stocks, securities, futures, and bonds. It should be noted that, in the embodiments of the present invention, any credit instrument that can serve as a certificate of economic rights for a user can be called a financial product, such as securities, bonds / derivative market products (such as funds or stocks, futures, options, interest rate futures, etc.). This application does not impose specific restrictions on which financial product is used.

[0153] Specifically, data on relevant funds or stocks can be obtained as historical parameters for financial scenarios based on the identifiers (such as codes) corresponding to individual stocks or the overall market. Using the text information processing model training method provided in this application, when attempting to obtain preference information for any fund or stock by parsing text information processing requests, the historical behavior parameters of the target object can be used to perform data cross-filtering within the historical parameters of the financial scenario to obtain textual descriptions of the market conditions and trading volume of any fund or stock. For example, the obtained textual information matching the target object could be: 1) "The XXX fund or stock is performing well, it's advisable to hold a long position; I have already bought XXX lots of XXX fund"; 2) "The XXX fund or stock has been performing poorly recently, but I have already sold all my shares."

[0154] It is understood that in the specific implementation of this application, user-related data such as the historical behavior parameters of the target object are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0155] In some embodiments, the text information processing model training device may collect historical parameters corresponding to funds or stocks through at least one of the following methods: obtaining fund or stock data corresponding to funds or stocks in a database; calling an application programming interface (API) to obtain fund or stock data corresponding to funds or stocks; or crawling historical data from web pages and historical behavior parameters of the target object (whether the fund or stock is selected as a watchlist or whether there are transaction records) through a web crawler.

[0156] For example, the text information processing model training device can be a server provided by an operator, equipped with a database for storing historical data on funds or stocks. This database stores data on multiple individual stocks over a period of time (e.g., one month) and overall market data over the same period. When the server needs to obtain individual stock data (or market data), it queries the database for the corresponding stock code. For example, when the server needs to obtain real-time data on funds or stocks, it calls an API to obtain the real-time data on the desired stock and the overall market data through a data interface with a stock exchange. For example, when the server needs to obtain individual stock data (or market data), it can use a web crawler to retrieve the corresponding fund or stock data from relevant external websites. In the process of operating financial products, user comments on a certain fund or stock, or transaction notes on a certain sector of funds or stocks, can be obtained as text information to be processed and matched with the target object. By analyzing the emotional state of the text information to be processed, funds or stocks (or a certain sector of funds or stocks) corresponding to the text information with a positive emotional state can be recommended to users in the financial mini-program.

[0157] Furthermore, considering that the proposed solution can be implemented not only through financial apps but also through instant messaging mini-programs in practical applications, by storing multi-level graph neural networks in a financial blockchain, users can quickly predict financial text information for different targets when changing terminals through text information processing models deployed in the blockchain network.

[0158] Specifically, the target object identifier, the model parameters of the text information processing model, and the target object identifier can be sent to the blockchain network, so that the nodes of the blockchain network can fill the target object identifier, the model parameters of the text information processing model, and the target object identifier into a new block, and when there is consensus on the new block, the new block is appended to the end of the blockchain.

[0159] In this invention, embodiments can be implemented using cloud technology. Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. It can also be understood as a general term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies based on cloud computing business models. The backend services of network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites; therefore, cloud technology needs cloud computing as its support.

[0160] It's important to note that cloud computing is a computing model that distributes computing tasks across a resource pool comprised of numerous computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" are infinitely scalable, readily available, and can be used on demand, expanded at any time, and paid for based on usage. As the foundational providers of cloud computing capabilities, they establish cloud resource pool platforms, often referred to as cloud platforms or Infrastructure as a Service (IaaS). These platforms deploy various types of virtual resources within the resource pool for external customers to choose from. The cloud resource pool primarily includes: computing devices (which can be virtualized machines containing operating systems), storage devices, and network devices.

[0161] Combined with the preceding sequence Figure 1 As shown, the text information processing model training method provided in this embodiment of the invention can be implemented through corresponding cloud devices. For example, terminals (including terminals 10-1 and 10-2) connect to a server 200 located in the cloud via a network 300. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. It is worth noting that the server 200 can be a physical device or a virtualized device.

[0162] Beneficial technical effects:

[0163] This invention, in response to a text information processing request, acquires text information and tag information; performs probability prediction processing on the text information using a text prediction network of a text processing model to obtain a text embedding feature vector and predicted probability distribution information; performs embedding processing on the tag information using a deep neural network of the text processing model to obtain a tag embedding feature vector; performs one-hot encoding processing on the tag information using a one-hot encoding network of the text processing model to obtain a one-hot encoded feature vector; calculates the actual probability distribution information based on the text embedding feature vector, the tag embedding feature vector, and the one-hot encoded feature vector; and adjusts the loss function of the text processing model based on the predicted probability distribution information and the actual probability distribution information to achieve a classification result for the text information to be processed. This enables the adjustment of the loss function of the text processing model by using predicted and actual probability distribution information, saving training time, improving classification accuracy, and enhancing the user experience.

[0164] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for training a text information processing model, characterized in that, The method includes: In response to a text information processing request, retrieve text information and tag information; The text information is subjected to probability prediction processing through the text prediction network of the text processing model to obtain the text embedding feature vector and prediction probability distribution information of the text information. Based on the index information of the label embedding feature vector, the number of dimensions of the label embedding feature vector is determined; Calculate the tag information matrix based on the number of tag types and the number of dimensions of the tag information; The tag information matrix is ​​embedded through the embedding layer network of the deep neural network of the text processing model to obtain the tag embedding feature vector of the tag information. The tag information is processed by one-hot encoding through the one-hot encoding network of the text processing model to obtain the one-hot encoded feature vector of the tag information. The product of the text embedding feature vector and the tag embedding feature vector is subjected to a nonlinear transformation to obtain the similarity between the text embedding feature vector and the tag embedding feature vector. Based on the similarity, the one-hot encoded feature vector is subjected to vector smoothing to obtain the actual probability distribution information. Based on the predicted probability distribution information and the actual probability distribution information, the loss function of the text processing model is adjusted to obtain the classification result of the text information to be processed through the text processing model. The process of obtaining the classification result of the text information to be processed through the text processing model includes: determining at least one word-level latent variable corresponding to the text information to be processed through the encoder of the text processing model; generating a classification result corresponding to the word-level latent variable and a selection probability of the classification result based on the at least one word-level latent variable through the decoder of the text processing model; and selecting at least one of the classification results based on the selection probability of the text classification result.

2. The method according to claim 1, characterized in that, The process of responding to a text information processing request and obtaining text information and tag information includes: Parse the text information processing request to determine the target object included in the text information processing request and the financial scenario corresponding to the target object; In the financial scenario, the historical behavioral parameters of the target object and the historical parameters of the financial scenario are determined; Based on the target object, the historical behavior parameters of the target object and the historical parameters of the financial scenario are cross-filtered to obtain text information that matches the target object; Based on the target object, the historical behavior parameters of the target object and the historical parameters of the financial scenario are cross-filtered to obtain tag information that matches the target object.

3. The method according to claim 1, characterized in that, The step of adjusting the loss function of the text processing model based on the predicted probability distribution information and the actual probability distribution information includes: Based on the predicted probability distribution information and the actual probability distribution information, the relative entropy of the text processing model is calculated; Based on the predicted probability distribution information and the one-hot encoded feature vector, the cross-entropy of the text processing model is calculated; The loss function of the text processing model is adjusted by replacing its cross-entropy with its relative entropy.

4. A text information processing method, characterized in that, The method includes: Obtain the text information to be processed; The encoder of the text processing model determines at least one word-level latent variable corresponding to the text information to be processed. The decoder of the text processing model generates a classification result corresponding to the word-level latent variable and the selection probability of the classification result based on the at least one word-level latent variable. Based on the probability of the classification result being selected, at least one classification result is selected; The text processing model is trained based on the method described in any one of claims 1 to 3.

5. A text information processing model training device, characterized in that, The device includes: The information transmission module is used to respond to text information processing requests and obtain text information and tag information; The information processing module is used to perform probability prediction processing on the text information through the text prediction network of the text processing model to obtain the text embedding feature vector and prediction probability distribution information of the text information. The information processing module is used to determine the number of dimensions of the tag embedding feature vector based on the index information of the tag embedding feature vector; calculate the tag information matrix of the tag information according to the number of types of the tag information and the number of dimensions; and perform embedding processing on the tag information matrix through the embedding layer network of the deep neural network of the text processing model to obtain the tag embedding feature vector of the tag information. The information processing module is used to perform one-hot encoding processing on the tag information through the one-hot encoding network of the text processing model to obtain the one-hot encoded feature vector of the tag information. The information processing module is used to perform nonlinear transformation processing on the product of the text embedding feature vector and the tag embedding feature vector to obtain the similarity between the text embedding feature vector and the tag embedding feature vector, and based on the similarity, to perform vector smoothing processing on the one-hot encoded feature vector to obtain the actual probability distribution information. The information processing module is used to adjust the loss function of the text processing model based on the predicted probability distribution information and the actual probability distribution information, so as to obtain the classification result of the text information to be processed through the text processing model. The process of obtaining the classification result of the text information to be processed through the text processing model includes: determining at least one word-level latent variable corresponding to the text information to be processed through the encoder of the text processing model; generating a classification result corresponding to the word-level latent variable and the selection probability of the classification result based on the at least one word-level latent variable through the decoder of the text processing model; and selecting at least one of the classification results based on the selection probability of the text classification result.

6. The apparatus as claimed in claim 5, characterized in that, The information processing module is also used for: The text information processing request is parsed to determine the target object included in the request and the corresponding financial scenario. Within the financial scenario, historical behavioral parameters of the target object and historical parameters of the financial scenario are determined. Based on the target object, cross-filtering processing is performed on the historical behavioral parameters of the target object and the historical parameters of the financial scenario to obtain text information matching the target object. Based on the target object, cross-filtering processing is performed on the historical behavioral parameters of the target object and the historical parameters of the financial scenario to obtain tag information matching the target object.

7. The apparatus as claimed in claim 5, characterized in that, The information processing module is also used for: Based on the predicted probability distribution information and the actual probability distribution information, the relative entropy of the text processing model is calculated; the relative entropy of the text processing model is used to replace the cross entropy of the text processing model in order to adjust the loss function of the text processing model, wherein the cross entropy is calculated based on the predicted probability distribution information and the one-hot encoded feature vector.

8. A text information processing device, characterized in that, The device includes: The data transmission module is used to acquire the text information to be processed. The data processing module determines at least one word-level latent variable corresponding to the text information to be processed through the encoder of the text processing model. The data processing module, through the decoder of the text processing model, generates a classification result corresponding to the latent variable at the word level and the probability of the classification result being selected, based on the at least one latent variable at the word level. The data processing module selects at least one classification result based on the selection probability of the classification result; The text processing model is trained based on the method described in any one of claims 1 to 3.

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the text information processing model training method according to any one of claims 1 to 3, or the text information processing method according to claim 4.

10. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; When a processor runs executable instructions stored in the memory, it implements the text information processing model training method according to any one of claims 1 to 4, or the text information processing method according to claim 5.

11. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the text information processing model training method according to any one of claims 1 to 3, or the text information processing method according to claim 4.

Images