Text processing method, device, electronic device and storage medium

Abstract

This disclosure provides a text processing method, apparatus, electronic device, and storage medium, relating to the field of data processing technology, particularly the field of deep learning technology and semantic matching technology. The method includes: acquiring a first text to be queried; performing feature processing on the first text to obtain a first feature vector; comparing the first feature vector with multiple second feature vectors to determine a target feature vector that matches the first feature vector; and determining a query result matching the first text based on the target feature vector. It is noteworthy that the first feature vector is a user-side feature, while the second feature vectors are query-side feature vectors. This disclosure allows for simultaneous processing of query-side and content-side features, enabling global processing and thus solving the technical problem of simultaneously ensuring semantic matching accuracy and processing efficiency, thereby improving the accuracy of the query results.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more particularly to the field of deep learning technology and semantic matching technology, specifically to a text processing method, apparatus, electronic device and storage medium. Background Technology

[0002] In many applications of NLP (Natural Language Processing), there is a need to measure text similarity semantically. We collectively refer to this need as semantic matching. Furthermore, semantic matching has a wide range of applications, such as in search, question-answering systems, and e-commerce. However, currently, there is no solution to the technical challenge of achieving good semantic matching results. Summary of the Invention

[0003] This disclosure provides a text processing method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of this disclosure, a text processing method is provided, comprising: obtaining a first text to be queried; performing feature processing on the first text to obtain a first feature vector; comparing the first feature vector with a plurality of second feature vectors to determine a target feature vector that matches the first feature vector, wherein the plurality of second feature vectors are obtained by performing feature cross calculation on at least two sets of feature vectors of the target card; and determining a query result that matches the first text based on the target feature vector.

[0005] According to a second aspect of this disclosure, a text processing apparatus is provided, comprising: an acquisition module for acquiring a first text to be queried; a feature processing module for performing feature processing on the first text to obtain a first feature vector; a comparison module for comparing the first feature vector with a plurality of second feature vectors to determine a target feature vector matching the first feature vector, wherein the plurality of second feature vectors are obtained by performing feature cross calculation on at least two sets of feature vectors of the target text; and a determination module for determining a query result matching the first text based on the target feature vector.

[0006] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the text processing method proposed in this disclosure.

[0007] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to perform the text processing method proposed in this disclosure.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that is executed by a processor using the text processing method proposed in this disclosure.

[0009] In embodiments of this disclosure, a text processing method, apparatus, electronic device, and storage medium that can be used for semantic matching are provided.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a flowchart of a text processing method according to an embodiment of the present disclosure;

[0013] Figure 2 This is a diagram showing the card information on the terminal's display interface;

[0014] Figure 3 This is a flowchart of a text processing method according to a preferred embodiment of the present disclosure;

[0015] Figure 4 This is a schematic diagram of the structure of a text processing apparatus according to an embodiment of the present disclosure;

[0016] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] In semantic matching techniques related to this technology, the following three solutions are typically used:

[0020] Option 1: Employ traditional text matching techniques, such as TF-IDF (Term Frequency-Inverse Document Frequency), BM25 (Best Matching 25), Jaccord, and SIMHash algorithms. The BM25 algorithm calculates a matching score based on the degree to which webpages cover query fields; higher scores indicate a better match. These traditional text matching techniques primarily address word-level matching, or similarity. However, they are limited to literal matching, resulting in less than ideal results. For example, "taxi" and "taxi" are dissimilar in wording but refer to the same mode of transportation; "apple" can mean different things in different contexts, such as fruit or company; "machine learning" and "learning machine" are identical words but have different meanings; and while "Qin Shi Huang plays Dota" is lexically and syntactically correct, it is incorrect when considering related knowledge.

[0021] Option 2: Employ unsupervised topic models within Latent Sementic Analysis (LSA) for semantic matching. This maps sentences to a low-dimensional continuous space of equal length, and then calculates similarity on this implicit latent semantic space. Examples include advanced probabilistic models such as PLSA (Probabilistic Latent Semantic Analysis) and LDA (Latent Dirichlet Allocation). However, Option 2 is essentially based on training with co-occurrence information, and the resulting technical performance cannot replace traditional text matching techniques; it can only serve as a supplement.

[0022] Option 3: Employing deep semantic matching techniques, including: Representation-based models, which extract the overall semantics of the text before matching, using a dual-tower shared parameter mapping to map the two texts to the same space to achieve meaningful matching; that is, encoding is performed at the table layer and interactive computation is performed at the matching layer, such as the Siamese structure; and Interaction-based models, which abandon the post-matching approach and perform word-to-word matching at the input layer, using the matching results as a grayscale image for subsequent modeling; that is, the interaction layer consists of an interaction matrix composed of words from both texts, and the representation layer is responsible for abstracting and representing the interaction matrix. However, while the representation-based models can preprocess the text and build an index, significantly reducing the time spent on online computation, they are prone to losing semantic focus and failing to measure the importance of word context, resulting in semantic drift. While interaction-based models can better grasp semantic intersections and model the importance of context, they tend to ignore global information such as syntax and inter-sentence comparisons, and cannot characterize global matching information from local matching information. In general, representation-based models are more efficient but have lower semantic matching accuracy, while interaction-based models have higher semantic matching accuracy, but due to more detailed modeling, they incur significantly higher computational costs, resulting in lower processing efficiency. Therefore, related technologies face the technical challenge of simultaneously ensuring semantic matching accuracy and processing efficiency.

[0023] To address the aforementioned technical issues, this disclosure provides a text processing method, apparatus, electronic device, and storage medium, which can be used in scenarios such as e-commerce, question-and-answer systems, dialogue systems, and machine translation where dynamic content recommendations can be made based on user needs.

[0024] Figure 1 This is a flowchart of a text processing method according to an embodiment of the present disclosure, such as... Figure 1 As shown, the method includes the following steps:

[0025] Step S101: Obtain the first text to be queried.

[0026] Specifically, the first text can be text information entered by the user, or a query that can represent the user's needs.

[0027] Step S102: Perform feature processing on the first text to obtain the first feature vector.

[0028] Specifically, the Baidu Erine model can be used to extract the embedding from the first text, obtaining the embedding corresponding to the first text. Then, the Poly Encoder model is used to process these embeddings, generating multiple embeddings based on each. These multiple embeddings are then dynamically integrated according to the second feature vector to obtain the first feature vector. It should be noted that the first feature vector can be obtained from the first text input by the user; therefore, the first feature vector is a user-side vector. In an optional embodiment, the first feature vector can be obtained using the following formula:

[0029]

[0030] w can be obtained using the following formula. i :

[0031]

[0032] Wherein, the first feature vector y q The second eigenvector y card Each embedding corresponding to the first text generates m embeddings, that is... wi is the attention of the i-th embedding among the m embeddings generated by the second feature vector and the embedding corresponding to the first text, where m is an integer greater than 0.

[0033] Furthermore, in actual deployment, the Poly Encoder model can calculate all second feature vectors offline. It only needs to calculate the embeddings corresponding to the first text online to generate m embeddings, and then quickly calculate the first feature vector through a simple dot product. Therefore, this disclosure can achieve the technical effect of improving processing efficiency.

[0034] Step S103: Compare the first feature vector with multiple second feature vectors to determine the target feature vector that matches the first feature vector. The multiple second feature vectors are obtained by performing feature cross calculation on at least two sets of feature vectors of the target card.

[0035] Specifically, the database used to generate information provided to users includes ID (Identity Document) data and text data, requiring the generation of corresponding embeddings for both. For ID data, the Transformer Encoder can be used to extract embeddings, generating embeddings corresponding to the ID data. For text data, the Baidu Erine model can be used to extract embeddings corresponding to the text data. Then, in the MOE Layer (Mixtures of Experts Layer), the embeddings corresponding to the ID data and the text data are feature-mixed to generate feature vectors for multiple candidate cards, i.e., the aforementioned second feature vectors. Next, the matching score between the first feature vector and the multiple second feature vectors is determined, and based on the matching score, the target feature vector corresponding to the first feature vector is determined from the multiple second feature vectors.

[0036] It should be noted that the above-mentioned text data belongs to the data to be queried and tested, which can be advertising data or text data required by users.

[0037] Step S104: Determine the query results that match the first text based on the target feature vector.

[0038] Specifically, the card information represented by the target feature vector is combined to generate query results, which are then displayed to the user who entered the first text.

[0039] In one optional embodiment, the text entered by the user can be a question about a disease, such as "What are the symptoms of gastrointestinal diseases?" The card information to be displayed to the customer includes a doctor card, a disease card, and an article card. The doctor card includes the doctor's name, photo, tags, and the diseases and diagnostic methods the doctor specializes in; the disease card includes the disease name, corresponding symptoms, and corresponding solutions; and the article card includes articles related to the disease, which may also include relevant figures. The target feature vector contains card vectors representing the above three types of card information. Based on the target feature vector, the corresponding card information is obtained and displayed to the user according to a preset display method, such as... Figure 2 As shown, Figure 2This is a diagram showing the card information displayed on the terminal's interface.

[0040] In the above embodiments of this disclosure, a first text to be queried is obtained; the first text is processed to obtain a first feature vector; the first feature vector is compared with multiple second feature vectors to determine a target feature vector that matches the first feature vector; and a query result matching the first text is determined based on the target feature vector. It is noteworthy that the first feature vector is a user-side feature, and the second feature vectors are query-side feature vectors. This disclosure can process query-side features and content-side features simultaneously, enabling global processing. This solves the technical problem of simultaneously ensuring semantic matching accuracy and processing efficiency, thereby improving the accuracy of the query results.

[0041] Optionally, feature processing is performed on the first text to obtain a first feature vector, including: performing feature processing on the first text to obtain multiple third feature vectors, wherein the multiple third feature vectors are encoded in different ways; and integrating the multiple third feature vectors to obtain the first feature vector.

[0042] Specifically, the third feature vector can be an embedding. This is achieved by extracting the embedding from the first text using the Baidu Erine model, obtaining an embedding corresponding to the first text, and then processing this embedding using the Poly Encoder model to generate multiple embeddings based on each first text-related embedding. This integrated processing can involve obtaining multiple third feature vectors based on a single first text, calculating the attention of each third feature vector, and then multiplying and summing the third feature vector with its corresponding attention to obtain the first feature vector. It should be noted that the third feature vectors can be generated using different encoders (i.e., transformers). In an optional embodiment, w i For the attention mechanism, let y be the i-th third feature vector generated from a single first text. q The third feature vector is The formula can be used: The first eigenvector is obtained.

[0043] In the above optional embodiments, multiple third feature vectors are integrated to obtain a first feature vector. Compared with the representational model, which uses only a single vector to represent the user-side feature vector, this can improve the technical effect of matching accuracy.

[0044] Optionally, the first text is subjected to feature processing to obtain multiple third feature vectors, wherein the multiple third feature vectors are encoded in different ways, including: using a first model to encode the features of the first text to obtain multiple third feature vectors, wherein the first model includes multiple feature encoders, and each feature encoder in the multiple feature encoders has a different encoding method.

[0045] Specifically, the first model can be a Poly Encoder model, which can generate multiple third feature vectors based on a single first text. The feature encoder can be a transformer; using a transformer in the Poly Encoder model, multiple third feature vectors can be generated by encoding based on a single first text. A Poly Encoder can include multiple transformers, and these transformers can be connected in parallel or in series. Furthermore, the transformer can also be a deep learning model based on a self-attention mechanism.

[0046] In the above optional embodiments, the first model can generate multiple third feature vectors based on a single first text. Compared with the representational model, which only uses a single vector to represent the user-side feature vector, this can achieve the technical effect of improving matching accuracy.

[0047] Optionally, the method further includes: obtaining at least one target card, wherein the target card includes at least identification information and text information, the identification information being used to describe the category of the target card and the text information being used to describe the content of the target card; performing feature extraction on at least the identification information and the text information respectively to obtain at least two sets of feature vectors of the target card; performing feature cross calculation on the at least two sets of feature vectors of the target card, and obtaining multiple second feature vectors based on the calculation results.

[0048] Specifically, the target card can be a form of content-side information storage. Each card contains corresponding text information and identifier information. The identifier information can be ID-type features, including but not limited to UserID, CategoryID, and CardID. UserID identifies the advertiser, CategoryID identifies the category to which the card belongs, and CardID identifies the card itself. The text information includes the content information on the card. For example, if the card is a doctor's card, the text information on the card could be introductory text introducing a doctor. A set of feature vectors can be based on the identifier information, using a Transformer Encoder to extract embeddings corresponding to the identifier information, or it can be based on the text information, using the Baidu Erine model to extract embeddings corresponding to the text information. Then, the above at least two sets of feature vectors are transmitted to the MOE Layer (Mixtures of Experts Layer). The MOE Layer performs feature mixing on the above at least two sets of feature vectors, re-memorizing the cards that appear frequently and re-generalizing the cards that appear infrequently or not at all, to obtain multiple candidate card vectors, namely the above second feature vector.

[0049] In the above optional embodiments, an expert hybrid model is used to perform feature mixing on a large amount of feature data, thereby achieving a high matching accuracy.

[0050] Optionally, feature crossing is performed on at least two sets of feature vectors of the target card, and multiple second feature vectors are obtained based on the calculation results, including: performing feature crossing on at least two sets of feature vectors using multiple second models to obtain multiple output results, wherein the multiple output results correspond to multiple second models; obtaining multiple target weight values ​​corresponding to the multiple output results; determining the target output result among the multiple output results based on the gating function and the multiple target weight values; and determining multiple second feature vectors based on the target output results.

[0051] Specifically, a set of feature vectors can be based on identifier information, using a Transformer Encoder to extract embeddings corresponding to the identifier information; or it can be based on text information, using the Baidu Erine model to extract embeddings corresponding to the text information. Then, these at least two sets of feature vectors are passed to the MOE Layer (Mixtures of Experts Layer), where feature mixing is performed to obtain multiple output results. It should be noted that the MOE Layer contains multiple expert models, and the second model can be an expert model within the MOE Layer. Each expert model processes at least one set of data, and multiple expert models process multiple sets of data, resulting in multiple output results, which include weight values. Then, a second feature vector is calculated based on a gating function and the weight values ​​in the output results.

[0052] In an alternative embodiment, the gating function can be a softmax function of dimension N. Furthermore, if the MOE structure is applied to a transformer, each layer of the FNN (Feed-forward Neural Network) module in the transformer (a Seq2Seq model) is replaced with multiple FNN modules as expert models, and gating is added.

[0053] In the above optional embodiments, an expert hybrid model is used to perform feature mixing on a large amount of feature data, thereby achieving the technical effect of improving matching accuracy.

[0054] Optionally, the method further includes: determining the query result matching the first text based on the target feature vector, including: using a third model to determine the query result corresponding to the first text based on the target feature vector, wherein the third model is trained by machine learning using training samples, and the training samples include: query sample data and query results corresponding to the query sample data.

[0055] Specifically, the third model can be a trained, mature matching model used to determine the target feature vector from multiple second feature vectors based on the first feature vector generated from the first text. The query result is the information represented by the target feature vector. Furthermore, training samples can be constructed using Listwise, and this training sample can be used to train the matching model, resulting in a trained, mature matching model, i.e., the third model. Then, the third model is used to determine the query result matching the first text.

[0056] In one optional embodiment, the query sample data can be text information or multiple card information. The query result is the card information that needs to be displayed to the customer. For example, the text information is "What are the symptoms of gastrointestinal diseases?", and the card information includes multiple doctor cards, disease cards, and article cards. The final query result is the doctor cards, disease cards, and article cards corresponding to "What are the symptoms of gastrointestinal diseases?". In the above optional embodiment, the matching model trained using Listwise is used to determine the query result of the first text, achieving the technical effect of improving matching accuracy.

[0057] Optionally, the method further includes: obtaining query sample data and query results corresponding to the query sample data, wherein the query results include at least one sample card; scoring at least one sample card in the query results to obtain a scoring result, wherein the scoring result is used to determine the sample card to be displayed among the at least one sample card; and updating the model parameters of the third model based on the scoring result.

[0058] In an optional embodiment, Listwise uses the list of all card results corresponding to each first text as a training example, i.e., the above sample data. The optimal scoring function F is trained based on the training example. Then, after receiving a new first text to be queried, the scoring function F is used to score all cards of the new first text to be queried, and the above scoring results are obtained.

[0059] In the above optional embodiments, the matching model trained by Listwise is used to determine the query result of the first text, thereby achieving the technical effect of improving the matching accuracy.

[0060] Optionally, updating the model parameters of the third model based on the scoring results includes: sorting the sample cards to be displayed based on the scoring results to obtain a sorting result, wherein the sorting result is used to determine the display order of each sample card in the sample cards to be displayed; and updating the model parameters of the third model based on the sorting result.

[0061] In one alternative embodiment, cards are sorted from highest to lowest score to obtain the final ranking result. That is, CVR (Conversion Rate) is used as the ranking metric in the card list, arranging the cards in the list from highest to lowest CVR. Furthermore, during the ranking process of this model, the normalized CVR values ​​in the list are used as labels, and the softmax value (softmax is a normalization function) of the prediction results is used as the prediction value. The KL divergence (Kullback-Leibler Divergence) between the two is optimized, while NDCG (Normailazed Discounted Cumulative Gain) is used as the optimization metric.

[0062] In the above optional embodiments, the technical effect of improving matching accuracy is achieved.

[0063] In a preferred embodiment of this disclosure, such as Figure 3 The above, Figure 3 The left side shows the processing on the user side. After obtaining the text to be queried from the user, EMB1, EMB2, ..., EMBn represent multiple embeddings generated based on the text to be queried. For each query, m (m is an integer greater than 0) embeddings are generated. Since multiple queries can be received from the user, multiple embeddings can be obtained. However, each time, only m embeddings generated based on one query are extracted from the multiple embeddings generated based on the query and transmitted to the attention module for processing. Figure 3 The right side represents the processing of the content, which includes ID-type information and text-type information. Figure 3 The UserID, CategoryID, and CardID shown are ID-type information, and CardText is text-type information. The embeddings for text-type information are extracted using the Baidu Wenxin model, while the EMB layer in the Transformer model extracts the embeddings for ID-type information. The extracted features are then passed to the MOE layer for feature mixing to obtain candidate card vectors, where FFN1, FFN2…FFN… n It can be multiple FFN modules in the transformer, where each FFN module identifies an Expert in the MOE. Figure 3The Gating function shown is the gate function. Then, the candidate card vectors are transmitted to the attention module. In the attention module, based on a query text, m embeddings and candidate card vectors are generated to obtain the corresponding attention value, i.e., Attention. Then, the user dynamic feature vector is obtained. Finally, based on the user dynamic feature vector and the candidate card vectors, the matching score between the selected card vector and the user feature vector is obtained.

[0064] According to the embodiments of this disclosure, an embodiment of a text processing device is also provided. The specific implementation methods and optional embodiments in this embodiment are similar to or the same as those in the above embodiments, and will not be described in detail here.

[0065] Figure 4 This is a schematic diagram of the structure of a text processing apparatus according to an embodiment of the present disclosure, such as... Figure 4 As shown, the device includes:

[0066] The acquisition module 41 is used to acquire the first text to be queried.

[0067] The feature processing module 42 is used to perform feature processing on the first text to obtain the first feature vector.

[0068] The comparison module 43 is used to compare the first feature vector with multiple second feature vectors to determine the target feature vector that matches the first feature vector. The multiple second feature vectors are obtained by performing feature cross calculation on at least two sets of feature vectors of the target card.

[0069] The determination module 44 is used to determine the query results that match the first text based on the target feature vector.

[0070] In the embodiments described above, a first text to be queried is obtained; the first text is processed to obtain a first feature vector; the first feature vector is compared with multiple second feature vectors to determine a target feature vector that matches the first feature vector; and a query result matching the first text is determined based on the target feature vector. It is noteworthy that the first feature vector is a user-side feature, while the second feature vectors are query-side feature vectors. This disclosure allows for simultaneous processing of query-side and content-side features, enabling global processing and thus solving the technical problem of simultaneously ensuring semantic matching accuracy and processing efficiency, thereby improving the accuracy of the query results.

[0071] Optionally, the feature processing module 42 includes:

[0072] The processing submodule is used to perform feature processing on the first text to obtain multiple third feature vectors, wherein the encoding methods of the multiple third feature vectors are different.

[0073] The integration submodule is used to integrate multiple third feature vectors to obtain the first feature vector.

[0074] Optionally, the processing submodule is also used to perform feature encoding on the first text using the first model to obtain multiple third feature vectors, wherein the first model includes multiple feature encoders, and each feature encoder in the multiple feature encoders has a different encoding method.

[0075] Optionally, the device further includes:

[0076] A card acquisition module is used to acquire at least one target card, wherein the target card includes at least identification information and text information, the identification information being used to describe the category of the target card, and the text information being used to describe the content of the target card.

[0077] The feature extraction module is used to extract features from at least the identification information and the text information to obtain at least two sets of feature vectors for the target card.

[0078] The feature cross module is used to perform feature cross calculations on at least two sets of feature vectors of the target card and obtain multiple second feature vectors based on the calculation results.

[0079] Optionally, the feature crossing module includes:

[0080] The first output submodule is used to perform feature crossing on at least two sets of feature vectors using multiple second models to obtain multiple output results, wherein the multiple output results correspond to multiple second models.

[0081] The weight acquisition submodule is used to obtain multiple target weight values ​​corresponding to multiple output results.

[0082] The target determination submodule is used to determine the target output result among multiple output results based on the gating function and multiple target weight values.

[0083] The vector determination submodule is used to determine multiple second feature vectors based on the target output.

[0084] Optionally, the determining module 44 is further configured to use a third model to determine the query results corresponding to the first text based on the target feature vector, wherein the third model is trained using training samples through machine learning, and the training samples include: query sample data and query results corresponding to the query sample data.

[0085] Optionally, the device further includes:

[0086] The query acquisition module is used to acquire query sample data and the corresponding query results, wherein the query results include at least one sample card.

[0087] The scoring module is used to score at least one sample card in the query results and obtain a scoring result, wherein the scoring result is used to determine the sample card to be displayed from at least one sample card.

[0088] The update module is used to update the model parameters of the third model based on the scoring results.

[0089] Optionally, the update module includes:

[0090] The sorting submodule is used to sort the sample cards to be displayed based on the scoring results, and obtain the sorting results. The sorting results are used to determine the display order of each sample card in the sample cards to be displayed.

[0091] The parameter update unit is used to update the model parameters of the third model based on the ranking results.

[0092] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0093] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0094] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0095] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0096] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform data processing methods by any other suitable means (e.g., by means of firmware).

[0097] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0099] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0101] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0102] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0103] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0104] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A text processing method, wherein, include: Get the first text to be queried; The first text is subjected to feature processing to obtain a first feature vector; The first feature vector is compared with a plurality of second feature vectors to determine a target feature vector that matches the first feature vector. The plurality of second feature vectors are obtained by performing feature cross calculation on at least two sets of feature vectors of the target card. The target card includes at least identification information and text information. The at least two sets of feature vectors are obtained by extracting features from at least the identification information and the text information respectively. Based on the target feature vector, a query result matching the first text is determined, wherein the query result is obtained by combining the card information represented by the target feature vector; The first text is subjected to feature processing to obtain a first feature vector, including: performing feature processing on the first text to obtain multiple third feature vectors, wherein the multiple third feature vectors are encoded in different ways; calculating the attention of each third feature vector; and multiplying and summing the multiple third feature vectors with the corresponding attention to obtain the first feature vector.

2. The method according to claim 1, wherein, The first text is subjected to feature processing to obtain multiple third feature vectors, wherein the encoding methods of the multiple third feature vectors are different, including: The first text is feature-encoded using a first model to obtain the plurality of third feature vectors. The first model includes a plurality of feature encoders, each of which uses a different encoding method.

3. The method according to claim 1, wherein, The method further includes: Obtain at least one target card, wherein the identification information is used to describe the category of the target card, and the text information is used to describe the content of the target card; At least two sets of feature vectors are obtained by extracting features from the identification information and the text information respectively to obtain the target card; Feature cross-calculation is performed on at least two sets of feature vectors of the target card, and the plurality of second feature vectors are obtained based on the calculation results.

4. The method according to claim 3, wherein, The target card undergoes feature cross-interaction with at least two sets of feature vectors, and the plurality of second feature vectors are obtained based on the calculation results, including: Multiple second models are used to perform feature crossing on at least two sets of feature vectors to obtain multiple output results, wherein the multiple output results correspond to the multiple second models; Obtain multiple target weight values ​​corresponding to the multiple output results; The target output result among the multiple output results is determined based on the gating function and the multiple target weight values; The plurality of second feature vectors are determined based on the target output.

5. The method according to claim 1, wherein, Determining the query result matching the first text based on the target feature vector includes: The third model is used to determine the query results corresponding to the first text based on the target feature vector. The third model is trained by machine learning using training samples, and the training samples include: query sample data and query results corresponding to the query sample data.

6. The method according to claim 5, wherein, The method further includes: Obtain the query sample data and the query results corresponding to the query sample data, wherein the query results include at least one sample card; At least one sample card in the query results is scored to obtain a score result, wherein the score result is used to determine the sample card to be displayed among the at least one sample card; The model parameters of the third model are updated based on the scoring results.

7. The method according to claim 6, wherein, The model parameters of the third model are updated based on the scoring results, including: Based on the scoring results, the sample cards to be displayed are sorted to obtain a sorting result, wherein the sorting result is used to determine the display order of each sample card in the sample cards to be displayed; The model parameters of the third model are updated based on the ranking results.

8. A text processing apparatus, wherein, include: The retrieval module is used to retrieve the first text to be queried; The feature processing module is used to perform feature processing on the first text to obtain a first feature vector; The comparison module is used to compare the first feature vector with a plurality of second feature vectors to determine a target feature vector that matches the first feature vector. The plurality of second feature vectors are obtained by performing feature cross calculation on at least two sets of feature vectors of the target card. The target card includes at least identification information and text information. The at least two sets of feature vectors are obtained by extracting features from at least the identification information and the text information respectively. The determining module is used to determine a query result that matches the first text based on the target feature vector, wherein the query result is obtained by combining the card information represented by the target feature vector; The feature processing module is further configured to perform feature processing on the first text through the following steps to obtain a first feature vector: perform feature processing on the first text to obtain multiple third feature vectors, wherein the multiple third feature vectors are encoded in different ways; calculate the attention of each third feature vector, and multiply and sum the multiple third feature vectors with the corresponding attention to obtain the first feature vector.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images