Transaction processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a transaction processing method and device, a storage medium and electronic device.The method comprises the steps that an initial transaction processing large model is created based on a basic large model, a sample user query prompt is input into the initial transaction processing large model for model training, and in the forward propagation model training process, a user query prompt is input into the initial transaction processing large model; the method comprises the following steps: performing transaction preprocessing on a sample user query prompt through an initial transaction processing large model to obtain a reference processing result and a sample reference query prompt, and performing transaction processing by adopting a cross attention mechanism based on the sample user query prompt and the sample reference query prompt to obtain a prediction processing result; and determining model loss based on the prediction processing result, the reference processing result, the sample user query prompt and the sample reference query prompt, and performing model parameter adjustment on the initial transaction processing large model based on the model loss to obtain the transaction processing large model.

Description

Technical Field

[0001] The present specification relates to the field of computer technology, and in particular to a transaction processing method, device, storage medium and electronic device. Background Art

[0002] With the rapid development of computer technology, transaction applications using large language models (LLMs) are becoming more and more widespread. Large language models have been widely used in public affairs, online shopping, consumer finance, government affairs and people's livelihood due to their excellent performance in natural language understanding and generation. In these transaction scenarios, large language models can provide efficient task processing and decision support based on natural language queries or requests input by users. Summary of the invention

[0003] This specification provides a transaction processing method, device, storage medium and electronic device, and the technical solution is as follows:

[0004] In a first aspect, this specification provides a transaction processing method, the method comprising:

[0005] Create an initial transaction processing model based on the basic model, and input sample user query prompts into the initial transaction processing model for model training;

[0006] In the forward propagation model training process, the sample user query prompt is subjected to transaction preprocessing by the initial transaction processing large model to obtain a reference processing result and a sample reference query prompt, and a cross-attention mechanism is used to perform transaction processing based on the sample user query prompt and the sample reference query prompt to obtain a prediction processing result;

[0007] During the back-propagation model training process, the model loss is determined based on the predicted processing results, the reference processing results, the sample user query prompts, and the sample reference query prompts, and the model parameters of the initial transaction processing large model are adjusted based on the model loss to obtain the transaction processing large model.

[0008] In a second aspect, this specification provides a transaction processing method, the method comprising:

[0009] Get the user query prompt entered by the user;

[0010] The user query prompt is input into a transaction processing big model, the user query prompt is subjected to transaction preprocessing by the transaction processing big model to obtain a reference query prompt, and transaction processing is performed based on the user query prompt and the reference query prompt using a cross-attention mechanism to obtain a transaction processing result.

[0011] In a third aspect, this specification provides a transaction processing device, the device comprising:

[0012] A model creation module is used to create an initial transaction processing model based on the basic model, and input sample user query prompts into the initial transaction processing model for model training;

[0013] A model training module, used for performing transaction preprocessing on the sample user query prompts by using the initial transaction processing large model to obtain reference processing results and sample reference query prompts during the forward propagation model training process, and performing transaction processing based on the sample user query prompts and the sample reference query prompts by using a cross-attention mechanism to obtain a prediction processing result;

[0014] The model training module is used to determine the model loss based on the prediction processing results, the reference processing results, the sample user query prompts and the sample reference query prompts during the back propagation model training process, and adjust the model parameters of the initial transaction processing large model based on the model loss to obtain the transaction processing large model.

[0015] In a fourth aspect, this specification provides a transaction processing device, the device comprising:

[0016] A prompt acquisition module is used to obtain the user query prompt input by the user;

[0017] A transaction processing module is used to input the user query prompt into a transaction processing large model, perform transaction preprocessing on the user query prompt through the transaction processing large model to obtain a reference query prompt, and perform transaction processing based on the user query prompt and the reference query prompt using a cross-attention mechanism to obtain a transaction processing result.

[0018] In a fifth aspect, the present specification provides a computer storage medium, wherein the computer storage medium stores at least one instruction, wherein the instruction is suitable for being loaded by a processor and executing the method steps of one or more embodiments of the present specification.

[0019] In a sixth aspect, the present specification provides a computer program product, wherein the computer program product stores at least one instruction, wherein the instruction is suitable for being loaded by a processor and executing the method steps of one or more embodiments of the present specification.

[0020] In a fifth aspect, the present specification provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps of one or more embodiments of the present specification.

[0021] The beneficial effects brought by the technical solutions provided by some embodiments of this specification include at least:

[0022] In one or more embodiments of the present specification, the electronic device inputs the sample user query prompt into the initial transaction processing large model based on the basic large model for model training. In the forward propagation model training process, the sample user query prompt is pre-processed by the initial transaction processing large model to obtain reference processing results and sample reference query prompts. Based on the sample user query prompts and the sample reference query prompts, a cross-attention mechanism is used to perform transaction processing to obtain a prediction processing result. In the reverse propagation model training process, the model loss is determined based on the prediction processing result, the reference processing result, the sample user query prompts and the sample reference query prompts. The model parameters of the initial transaction processing large model are adjusted based on the model loss to obtain a transaction processing large model. Thus, the response accuracy of the large model in the transaction processing scenario and the ability to align with user needs are gradually improved. The model training optimization process effectively reduces the generation of hallucinations and deviations from user requests, so that the output of the transaction processing model applied in the transaction scenario is more in line with the user's transaction processing needs, thereby significantly enhancing the application effect and user experience of the large language model in complex transaction scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in this specification or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 It is a scenario diagram of a transaction processing system provided in this specification;

[0025] Figure 2 It is a flowchart of a transaction processing method provided in this specification;

[0026] Figure 3 This is a flowchart of a forward propagation model training provided in this manual;

[0027] Figure 4 It is a model scenario diagram of a large transaction processing model provided in this specification;

[0028] Figure 5 This is a flowchart of a prompt alignment process provided in this manual;

[0029] Figure 6 It is a schematic diagram of the processing process of a prompt optimization module provided in this specification;

[0030] Figure 7This is a flowchart of a transaction processing based on a cross-attention mechanism provided in this specification;

[0031] Figure 8 This is a flowchart of a back propagation model training provided in this manual;

[0032] Fig. 9 It is a flowchart of a transaction processing method provided in this specification;

[0033] Fig.10 It is a structural schematic diagram of a transaction processing device provided in this specification;

[0034] Fig.11 It is a structural schematic diagram of another transaction processing device provided in this specification;

[0035] Fig.12 It is a structural schematic diagram of an electronic device provided in this manual. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in this specification to clearly and completely describe the technical solutions in this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0037] In the description of this specification, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in this specification can be understood in specific circumstances. In addition, in the description of this specification, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0038] In related technologies, large language models have shown excellent performance in natural language processing (NLP) tasks, but they still face problems such as hallucination generation, difficulty in strictly following user requests, and inaccurate responses, which has led to the phenomenon that large language models are not effective in transaction processing in transaction scenarios and cannot fully align with user needs. The existence of these phenomena affects the user experience and limits the application potential of LLM in complex transaction processing. Therefore, how to optimize the output generated by large language models in transaction scenarios so that it is more in line with user intentions has become the focus of current research.

[0039] The present specification is described in detail below with reference to specific embodiments.

[0040] See also Figure 1 , is a scenario diagram of a transaction processing system provided in this specification. Figure 1 As shown, the transaction processing system may include at least a client cluster and a service platform 100 .

[0041] The client cluster may include at least one client, such as Figure 1 As shown, it specifically includes client 1 corresponding to user 1, client 2 corresponding to user 2, ..., client n corresponding to user n, where n is an integer greater than 0.

[0042] Each client in the client cluster can be an electronic device with communication function, including but not limited to: wearable device, handheld device, personal computer, tablet computer, vehicle-mounted device, smart phone, computing device or other processing device connected to wireless modem, etc. Electronic devices can be called different names in different networks, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, electronic device, wireless communication device, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), electronic device in 5G network or future evolution network, etc.

[0043] The service platform 100 can be a separate server device, such as a rack-mounted, blade, tower, or cabinet-mounted server device, or a workstation, mainframe computer, or other hardware device with strong computing power; it can also be a server cluster composed of multiple servers, and the servers in the service cluster can be composed in a symmetrical manner, wherein each server has equivalent functions and status in the transaction link, and each server can provide services to the outside independently, and the independent service can be understood as not requiring the assistance of other servers.

[0044] In one or more embodiments of the present specification, the service platform 100 may establish a communication connection with at least one client in the client cluster, and complete data interaction during the transaction processing based on the communication connection, such as online transaction data interaction. For example, the service platform 100 may use the transaction processing model obtained based on the transaction processing method of the present specification to recommend content to the client. For example, the service platform 100 may obtain training data from the client, such as sample user query prompts.

[0045] It should be noted that the service platform 100 and at least one client in the client cluster establish a communication connection through a network for interactive communication, wherein the network can be a wireless network or a wired network, the wireless network includes but is not limited to a cellular network, a wireless local area network, an infrared network or a Bluetooth network, and the wired network includes but is not limited to Ethernet, a universal serial bus (USB) or a controller local area network. In one or more embodiments of the specification, technologies and / or formats including HyperText Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data (such as a target compressed package) exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0046] The transaction processing system embodiment provided in this specification and the transaction processing method in one or more embodiments belong to the same concept. The execution subject corresponding to the transaction processing method involved in one or more embodiments of the specification can be the above-mentioned service platform 100; the execution subject corresponding to the transaction processing method involved in one or more embodiments of the specification can also be the electronic device corresponding to the client, which is determined based on the actual application environment. The embodiment of the transaction processing system and its implementation process can be detailed in the following method embodiment, which will not be repeated here.

[0047] based on Figure 1 The scenario diagram is shown, and the transaction processing method provided by one or more embodiments of this specification is introduced in detail below.

[0048] See also Figure 2, a flowchart of a transaction processing method is provided for one or more embodiments of the present specification to train a transaction processing large model. The method can be implemented by a computer program and can be run on a transaction processing device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application. The transaction processing device can be an electronic device.

[0049] Specifically, the transaction processing method includes:

[0050] S102: creating an initial transaction processing large model based on the basic large model, and inputting sample user query prompts into the initial transaction processing large model for model training;

[0051] Basic large models: can be understood as pre-trained large language models, such as GPT-4, BERT, Tongyi Qianwen, etc. These basic large models are usually pre-trained on large-scale general data sets and have strong natural language understanding and generation capabilities, but lack the expertise required in specific fields (such as transaction processing).

[0052] Sample user query prompts: refers to the sample input data used to train the initial transaction processing model, which represents the queries that users may ask in real transaction processing scenarios. For example, users may enter "How do I apply for unemployment benefits?", "What should I do if my credit card is stolen?", "How can cross-training benefit groups such as runners, swimmers, or weightlifters?" The number of sample user query prompts can be multiple, and can include various transaction queries that users may ask, to help the large model learn how to identify user intent and provide appropriate transaction responses.

[0053] Optionally, a trained basic large language model can be obtained, and the basic large language model can be adapted to the transaction processing scenario using the methods of one or more embodiments of the present specification. Usually, the basic large language model is directly applied to dialogue responses in transaction processing scenarios, which is difficult to adapt to new transaction processing scenarios. Based on this, the basic large language model is first obtained to create an initial transaction processing large model, and new sample data for the transaction processing scenario - sample user query prompts are obtained. The sample user query prompts are sample query question data query in the transaction processing scenario. The sample user query prompts are used to perform model fine-tuning training on the initial transaction processing large model. After the model fine-tuning training is completed, a transaction processing large model adapted to the transaction processing scenario is obtained.

[0054] S104: in the forward propagation model training process, the sample user query prompt is subjected to transaction preprocessing by the initial transaction processing large model to obtain a reference processing result and a sample reference query prompt, and a cross-attention mechanism is used to perform transaction processing based on the sample user query prompt and the sample reference query prompt to obtain a prediction processing result;

[0055] Forward propagation: A step in the initial transaction processing model training process. The initial transaction processing model generates output (prediction results) based on the input sample data without adjusting the model parameters. After forward propagation, the loss is calculated and the parameters of the initial transaction processing model are updated through back propagation.

[0056] Transaction preprocessing: Before the initial transaction processing big model generates a formal answer to the sample user query prompt, it can directly process the user's input query based on the current big model part (its own basic big model module or a separate basic big model) to generate a reference processing result for it, and reversely infer the "query prompt" based on the reference processing result to generate a sample reference query prompt (equivalent to reversely predicting the query prompt based on the processing result).

[0057] Reference processing results: The target output generated based on the current large model part (its own basic large model module or a separate basic large model) during the transaction preprocessing process of the initial transaction processing large model, which is used as the reference answer (or target answer) in the formal transaction processing process of the initial transaction processing large model. The reference processing results help reversely predict sample reference query prompts.

[0058] Optionally, during the transaction preprocessing process, multiple reference processing results can be generated for the same sample user query prompt to achieve the effect of data enhancement on the original data, and multiple different versions of answers can be generated based on the current large model part (its own basic large model module or a separate basic large model) to enrich the training data and improve the robustness of the model.

[0059] Sample reference query prompt: As the reference input aligned with the sample user query prompt in the formal transaction processing process of the initial transaction processing big model, it is usually the input prompt predicted based on the reference processing result, which is used to guide the model to generate more accurate predictions and supervise the initial transaction processing to provide the correct prompts to generate better responses, so that the big model can be better aligned in the formal transaction processing process to meet the user's preferences and requests.

[0060] As a reverse-predictive optimized input query prompt based on the reference processing results, the sample reference query prompt can contain more contextual information, improved expressions, and more specific guidance, thereby helping the initial transaction processing model to better understand the user's intentions.

[0061] Cross-Attention Mechanism: The CrossAttention mechanism enables the initial transaction processing model to refer to another input (such as a sample reference query prompt) when processing one input (such as a user query prompt), thereby enhancing the model's understanding of the relationship between the two inputs.

[0062] Schematically, when predicting the processing results, the initial transaction processing large model takes the sample user query prompt and the sample reference query prompt as input, and uses the cross-attention mechanism to simultaneously refer to the sample user query prompt and the sample reference query prompt. The cross-attention mechanism allows the model to focus on different parts of the sample user query prompt and the sample reference query prompt when generating predictions, thereby improving the accuracy of transaction processing.

[0063] For example: the user query prompt is "How to apply for a tax refund?", and the reference query prompt can be "Query the documents and procedures required for a tax refund".

[0064] For example, if the user query prompt is "How can cross-training benefit groups such as runners, swimmers, or weightlifters?", the pre-generated reference processing results can be "Cross-training is a good way for athletes to improve their overall health, reduce the risk of injury, and improve their performance in their primary sport. Here are some ways that cross-training can benefit groups such as runners, swimmers, and weightlifters...", and the predicted reference query prompts can be:

[0065] “Please describe specific benefits of cross-training, such as how it can help runners, swimmers, or weightlifters improve their performance and reduce their risk of injury.”

[0066] “What are some ways to cross-train that can help a runner, swimmer, or weightlifter improve fitness, reduce injuries, and increase performance?”

[0067] “How can cross-training help different types of athletes in terms of improving overall fitness, reducing risk of sports injuries, and enhancing performance in their primary sport?” ......

[0069] These reference query prompts provide more context and details, helping the big model understand the specific issues that users care about (such as improving health, reducing injury risk, and improving athletic performance), thereby generating answers that are more in line with user intent.

[0070] Furthermore, through the cross-attention mechanism, the large model can pay attention to the user's query prompts and reference query prompts at the same time during forward propagation, so as to generate transaction processing answers more accurately. For example, the prediction processing result can be "Cross-training is an excellent way for athletes to improve their overall health, reduce the risk of injury and enhance their performance in major sports." After understanding the user's intention, the large model can generate a suitable and accurate prediction processing result. This prediction processing result not only describes the advantages of cross-training, but is also highly relevant to the core needs of the user's question.

[0071] S106: During the back propagation model training process, the model loss is determined based on the prediction processing result, the reference processing result, the sample user query prompt and the sample reference query prompt, and the model parameters of the initial transaction processing large model are adjusted based on the model loss to obtain the transaction processing large model.

[0072] Model loss: The loss function measures the difference in model output. The larger the loss, the greater the deviation in model output. In back propagation, the model optimizes parameters by minimizing the loss function.

[0073] Through back-propagation model training, the difference between the predicted processing results and the reference processing results is used as the main basis. At the same time, the reference sample user query prompts and sample reference query prompts are used to comprehensively calculate the model loss. The back-propagation algorithm is used to calculate the gradient of the model loss with respect to one or more model parameters, and the adjustment direction and size of the model parameters to be adjusted are determined according to the gradient. During the back-propagation process, the model will automatically adjust parameters (such as weights and biases) to reduce model loss until the initial transaction processing large model meets the model end training conditions, and the transaction processing large model after training is obtained, so that in future predictions, it can be ensured that the large model can better consider the original user intention in the loss function.

[0074] Optionally, the model training end condition of the model may include, for example, a model loss value is less than or equal to a preset loss function threshold, the number of iterations reaches a preset number threshold, etc. The specific model training end condition can be determined based on actual conditions and is not specifically limited here.

[0075] In one or more embodiments of the present specification, the electronic device inputs the sample user query prompt into the initial transaction processing large model based on the basic large model for model training. In the forward propagation model training process, the sample user query prompt is pre-processed by the initial transaction processing large model to obtain reference processing results and sample reference query prompts. Based on the sample user query prompts and the sample reference query prompts, a cross-attention mechanism is used to perform transaction processing to obtain a prediction processing result. In the reverse propagation model training process, the model loss is determined based on the prediction processing result, the reference processing result, the sample user query prompts and the sample reference query prompts. The model parameters of the initial transaction processing large model are adjusted based on the model loss to obtain a transaction processing large model. Thus, the response accuracy of the large model in the transaction processing scenario and the ability to align with user needs are gradually improved. The model training optimization process effectively reduces the generation of hallucinations and deviations from user requests, so that the output of the transaction processing model applied in the transaction scenario is more in line with the user's transaction processing needs, thereby significantly enhancing the application effect and user experience of the large language model in complex transaction scenarios.

[0076] Optionally, the (initial) transaction processing model includes at least a basic large model module (shown as "LLM"), a prompt optimization module and a cross-attention module, see Figure 3 , Figure 3 This is a flow chart of a forward propagation model training process proposed in one or more embodiments of this specification. Specifically, the sample user query prompt is pre-processed by the initial transaction processing large model to obtain a reference processing result and a sample reference query prompt, and a cross-attention mechanism is used to perform transaction processing based on the sample user query prompt and the sample reference query prompt to obtain a prediction processing result. The following method can be referred to:

[0077] S202: pre-processing the sample user query prompt by the basic large model module to obtain a reference processing result;

[0078] See also Figure 4 , Figure 4 The invention is a model scenario diagram of a transaction processing big model, wherein the (initial) transaction processing big model at least comprises a basic big model module (the "LLM" shown in the diagram), a prompt optimization module and a cross attention module (the "cross attention" shown in the diagram); the basic big model module is constructed using the basic big model, and the basic big model can be used as the basic big model module when creating the (initial) transaction processing big model. During the model training process, the model structure and parameters of the original basic big model of the basic big model module can be in a frozen state, and in the frozen state, the model parameters of the original basic big model in the basic big model module remain unchanged;

[0079] Self-supervision is achieved through data augmentation via LLM.

[0080] First, the prompt is fed into the LLM to create the desired augmentation, and then the initial prompt is used as the target and the text generated by the LLM is used as input. Therefore, by using only the prompt as input, self-supervised training is achieved. This allows access to a large amount of data during model training, as the only input required is a prompt. The key to this self-supervised strategy is that large models can be trained without external labeled data, thereby making full use of the existing LLM generation capabilities to enhance the dataset. This not only simplifies the data preparation process, but also greatly improves the training efficiency and effectiveness of the model. In this way, a variety of rich prompts can be generated more flexibly, and the LLM can be further aligned to generate outputs that meet user preferences and requests.

[0081] According to some embodiments, before the initial transaction processing big model generates a formal answer to the sample user query prompt, it can directly process the query input by the user based on the current basic big model module to generate a reference processing result for it.

[0082] S204: The prompt optimization module performs prompt alignment processing based on the sample user query prompt and the reference query answer to obtain a sample reference query prompt;

[0083] The prompt optimization module optimizes and aligns user query prompts to ensure that the output generated by the model better meets the needs of users. By learning the context of user queries and the characteristics of reference query answers, the user's initial prompt (sample user query prompt) is adjusted to obtain a sample reference query prompt, which is more in line with the model's understanding and facilitates the generation of more accurate responses.

[0084] Optionally, the prompt optimization module can be a self-supervised pre-training module based on optimal transfer, referred to as a POAT module, which combines the user query prompt and the reference query answer through the prompt optimization module, and uses back-propagation fine-tuning methods such as optimal transfer (OT) and cross-entropy loss to align the user query prompt with the reference query answer in the latent space, and then reversely predict the query prompt based on the reference query answer to obtain a sample reference query prompt. This makes the optimized sample reference query prompt more consistent with the understanding of the big model in terms of content and expression, thereby facilitating the big model to generate output that meets user needs.

[0085] Through alignment, the POAT module is able to generate an optimized sample reference query prompt, which makes the prompt more consistent with the model's understanding in terms of content and expression, making it easier for the model to generate output that meets the requirements.

[0086] Optional, please refer to Figure 5 , Figure 5The present invention is a flowchart of prompt alignment processing. Specifically, the prompt optimization module performs prompt alignment processing based on the sample user query prompt and the reference query answer to obtain the sample reference query prompt. The following method can be referred to:

[0087] S3002: The prompt optimization module performs embedding processing based on the sample user query prompt and the reference query answer to obtain a sample original query prompt embedding and a sample reference answer embedding;

[0088] The role of the prompt optimization module POAT is to act as an optimization prompt generator. Through the alignment strategy and generation mechanism learned in the training phase, POAT can generate a more optimized prompt when the user enters a simple query. This prompt can guide the transaction processing model to generate the required content more accurately.

[0089] like Figure 6 As shown, Figure 6 is a schematic diagram of the processing process of the prompt optimization module, which is preliminarily processed by the basic large model module ( Figure 6 The LLM shown in the figure pre-processes the sample user query prompt to obtain a reference processing result, that is, first use the ability of LLM to generate one or more corresponding answer reference processing results for each sample user query prompt for one or more existing sample user query prompts, perform sample data enhancement, and then embed the original query (sample user query prompt) and the generated reference query answer through the "LLM's embedding layer" (which can be called the embedding coding layer) in the prompt optimization module to generate prompt embeddings respectively, and obtain the sample original query prompt embedding and the sample reference answer embedding.

[0090] Exemplarily, the original query (sample user query prompt) and the generated reference query answer are embedded and coded through the "LLM's embedding layer" (which can be called the embedded coding layer) in the prompt optimization module. The embedded coding process can be a rotational position coding RoPE method. Rotational position coding is a position coding strategy that implements a rotation transformation through complex multiplication and integrates the position information into the token representation. RoPE is a fixed absolute position coding strategy, but it uses a rotation transformation to enable the inner product operation of Attention to automatically perceive the relative position information, thereby achieving the effect of relative position coding.

[0091] Optionally, a novel approach is proposed based on the prompt optimization module to infer the prompt by providing a corresponding generated answer to the large model. In addition to fine-tuning the model, emphasis is also placed on how to provide the right prompt to generate better responses; the prompt optimization module follows the adversarial variational autoencoder architecture, where the embedding layer of the large model is frozen from the LLM and shared across all inputs. The encoder, decoder, and discriminator of the prompt optimization module all use multi-head attention as the backbone. The encoder first converts the generated answer into a latent representation that is aligned with the latent representation of the prompt, and then the decoder decodes the latent representation into the prompt. In this way, the large model is able to understand how to word the prompt to best align the LLM to meet the user's preferences and requests.

[0092] S3004: performing encoding conversion processing on the sample original query hint embedding to obtain a sample original hint latent space, and performing encoding alignment processing on the sample reference answer embedding based on the sample original hint latent space to obtain a sample reference answer latent space;

[0093] Latent space is a low-dimensional space used to represent data in machine learning. It refers to a compressed representation of all the useful information contained in the data. The latent space is usually lower in dimensionality than the original data space, which makes it easier to analyze and understand.

[0094] The sample original prompt latent space is the vector representation of the original prompt in the latent space, and the sample original query prompt embedding is mapped into a latent space to better align and optimize the query prompt;

[0095] The sample reference answer latent space is to generate a vector representation of the reference answer in the latent space, and embed the sample reference answer into a latent space to better align and optimize the query prompt;

[0096] Furthermore, the generated embedding is encoded through the Encoder layer to generate a latent space (latent space representation). Specifically, the sample original query prompt embedding is encoded and converted to obtain the sample original prompt latent space (latent space), and the sample reference answer embedding is encoded and aligned based on the sample original prompt latent space to obtain the sample reference answer latent space (latent space). That is, using the information of the sample original prompt latent space, the sample reference answer embedding is encoded and aligned through the prompt optimization module, and it is also mapped to the same latent space as the sample original prompt latent space, and the sample reference answer latent space is represented in the same latent space, and the two latent spaces are distinguished by the discriminator, so that the two embedding representations are closer in the latent space, thereby achieving encoding alignment.

[0097] S3006: Decode the sample reference answer latent space to obtain a sample reference query prompt.

[0098] The decoding process is to convert the aligned sample reference answer latent space (i.e., the vector representation in the latent space) back to natural language text to generate the final sample reference query prompt.

[0099] The sample reference query prompt is the decoded prompt text, which is an alignment-optimized version that can better guide the model to generate answers that meet user needs.

[0100] Schematically, a decoder is used to decode the sample reference answer latent space (aligned vector representation) and convert it into a natural language prompt, thereby obtaining a sample reference query prompt, which, after being optimized by the POAT module, can more accurately guide the model to generate answer prompt text.

[0101] In one or more embodiments of the present specification, the prompt optimization module POAT uses embedding, encoding alignment, decoding and other processing methods to align user queries and reference queries to generate optimized sample reference query prompts. This process ensures that the sample reference query prompts are clearer and more structured, which can effectively guide the model to understand user needs and generate transaction processing results that are more in line with user intentions.

[0102] S206: The cross-attention module uses a cross-attention mechanism to perform transaction processing based on the sample user query prompt and the sample reference query prompt to obtain a prediction processing result.

[0103] The cross-attention module enables the large model to simultaneously refer to the user's original query prompt and the sample reference query prompt optimized by POAT, so as to comprehensively consider the information of the two prompts when generating the prediction processing results. The cross-attention mechanism (Cross-Attention Mechanism) is adopted by the cross-attention module to enable the large model to take into account both the user's intention and the expression of the reference answer when generating answers, thereby improving the accuracy of the generated results.

[0104] Exemplary, cross-attention mechanism: The cross-attention module uses cross-attention processing on the sample user query prompt and the sample reference query prompt. The cross-attention module calculates the attention distribution from one input sequence - "original query prompt", but refers to the information of another input sequence - "sample reference query prompt" when generating attention weights. The cross-attention mechanism usually uses three important matrices: query, key, and value. In the self-attention mechanism (Self-Attention), these three matrices come from the same input sequence; in cross-attention, the query comes from the first input sequence, and the key and value come from the second input sequence. The specific process is as follows:

[0105] Query, key and value calculation: The first input sequence (usually the sequence to be processed) is converted into a query vector (Query). The second input sequence (usually the auxiliary sequence for reference) is converted into a key vector (Key) and a value vector (Value).

[0106] Calculate attention weights: Use the query vector and key vector to calculate the similarity between each element. The higher the similarity, the more "attention" the query pays to the corresponding value.

[0107] Specifically, the similarity is usually calculated by dot product, and then normalized by Softmax to obtain the attention weight.

[0108] Generate weighted output: Use the calculated attention weights to perform a weighted average of the value vectors to generate an output. This output incorporates information from the second input sequence, allowing the model to consider the contents of both inputs when generating the result.

[0109] For example, suppose that in an (initial) transaction processing model, the (sample) user query prompt entered is "How to apply for unemployment benefits?", and the (sample) reference query prompt after POAT optimization is "Please describe the process and required materials for applying for unemployment benefits."

[0110] When processing transactions through the cross-attention mechanism: Convert sample user query prompts into query vectors. Convert optimized sample reference query prompts into key vectors and value vectors. Calculate attention weights: Calculate similarity between query and key, and generate attention weights so that the model can pay more attention to information that matches the user query content. Generate weighted output: Use attention weights to weight the sum of value vectors to generate a prediction output that combines the user query prompt and reference prompt information, and finally generate an accurate answer that meets the user's needs.

[0111] In this specification, the cross-attention mechanism of the cross-attention module helps large models better understand complex, multi-level task requirements by referring to the information of another input sequence when processing one input sequence. In transaction processing scenarios, cross-attention allows the model to pay attention to the user's actual query and optimized reference prompts when generating answers, thereby improving the accuracy and relevance of the output.

[0112] Optional, please refer to Figure 7 , Figure 7 The invention is a flow chart of transaction processing based on a cross-attention mechanism, and specifically, the cross-attention module uses the cross-attention mechanism to process transactions based on the sample user query prompt and the sample reference query prompt to obtain a prediction result, and the following implementation methods can be adopted:

[0113] S4002: Determine a sample original query prompt embedding corresponding to the sample user query prompt, and determine a sample reference query prompt embedding corresponding to the sample reference query prompt;

[0114] S4004: The cross-attention module uses a cross-attention mechanism to perform transaction processing based on the sample original query prompt embedding and the sample reference query prompt embedding to obtain a prediction processing result.

[0115] Cross-attention module: Using the cross-attention mechanism, the large model can simultaneously focus on the embedded representations of user queries and reference queries when generating transaction processing results.

[0116] Cross-attention mechanism for transaction processing: By embedding the sample original query prompt as the query and the sample reference query prompt as the key and value, the large model can combine the information of both to generate more accurate transaction processing results.

[0117] Exemplarily, the sample original query prompt embedding is input into the cross-attention module as a query vector, and the sample reference query prompt embedding is input into the cross-attention module as a key and value. The similarity between the sample original query prompt embedding and the sample reference query prompt embedding is calculated through the cross-attention mechanism to generate attention weights to determine the model's attention to different information. According to the attention weights, the embeddings of the reference query prompts are weighted and summed to generate a prediction processing result. The prediction processing result integrates the key information of the user query and the semantic information of the reference prompt, so that the large model can provide output that better meets user needs.

[0118] Schematically, the cross-attention module uses the embedded representation of sample user query prompts and sample reference query prompts, and adopts the cross-attention mechanism to generate prediction results for transaction processing. This process ensures that the model comprehensively considers the user's actual needs and the optimized reference prompts when generating answers, thereby generating high-quality transaction processing results that are more in line with user intentions.

[0119] In one or more embodiments of the present specification, through embedding processing and encoding alignment, the large model forms a clearer semantic understanding of the user's original query prompt and reference answer, so that the large model can more accurately identify the key information of the user's needs; and the prompt alignment processing of the POAT module optimizes the user's query prompt to make it more in line with the model generation requirements, thereby improving the accuracy and relevance of the generated results; and the cross-attention mechanism combines the sample user query and the optimized reference prompt information to generate a prediction processing result that is more in line with actual needs. This ensures that the answers output by the large model are not only comprehensive, but also closely linked to user needs; and, through alignment and decoding processing in the latent space (S206), the model can better align the user's initial needs during generation, reduce errors, and generate content that is easier to understand and meets transaction needs. Overall, the model's adaptability, generation accuracy, and ability to align user intent in complex transaction scenarios are improved, thereby significantly optimizing the user experience.

[0120] Optionally, according to some embodiments, the initial transaction processing model may include a basic model module, a prompt optimization module and a cross-attention module.

[0121] See also Figure 8 , Figure 8 The invention is a flowchart of back propagation model training, wherein the model loss is determined based on the prediction processing result, the reference processing result, the sample user query prompt and the sample reference query prompt, and the model parameters of the initial transaction processing model are adjusted based on the model loss. The following methods can be referred to:

[0122] S5002: Acquire a sample original prompt latent space corresponding to the sample user query prompt and a sample reference query prompt latent space corresponding to the sample reference query prompt;

[0123] According to some embodiments, during the forward propagation model training process, the prompt optimization module performs embedding processing based on the sample user query prompt and the reference query answer to obtain the sample original query prompt embedding and the sample reference answer embedding, performs encoding conversion processing on the sample original query prompt embedding to obtain the sample original prompt latent space, and performs encoding alignment processing on the sample reference answer embedding based on the sample original prompt latent space to obtain the sample reference answer latent space, that is, each round can be obtained from the forward propagation model training process: the sample user query prompt p t The corresponding sample original prompt latent space P, and the sample reference query prompt The corresponding sample reference query hints the latent space

[0124] S5004: Calculate the optimal transmission distance loss and relative entropy loss based on the sample original prompt latent space and the sample reference query prompt latent space, calculate the prompt optimization loss based on the sample user query prompt and the sample reference query prompt, adjust the parameters of the prompt optimization module based on the optimal transmission distance loss, relative entropy loss and the prompt optimization loss, and keep the parameters of the basic large model module unchanged;

[0125] Optimal Transmission Distance Loss (OT Loss): measures the transmission cost between the sample original hint latent space and the sample reference query hint latent space, ensuring that the user query and the reference query remain consistent in the latent space.

[0126] Schematically, the optimal transfer distance loss is calculated: the distance between the sample original prompt latent space and the sample reference query prompt latent space is calculated by the optimal transfer (OT) method, and the transfer cost is obtained as the optimal transfer distance loss, which is used to align the latent representations of the two prompts.

[0127] Relative entropy loss (KL loss): used to measure the difference between two distributions (such as the distribution of the original prompt of the sample and the reference prompt in the latent space) to ensure that the distribution of the model is consistent with the reference distribution.

[0128] Schematically, calculate the relative entropy loss: the distribution difference between the two latent space representations is calculated by relative entropy (KL divergence) to measure the distribution similarity of the sample original prompt and the sample reference prompt in the latent space. Minimizing the relative entropy loss ensures a more consistent distribution in the latent space, making the generated content more consistent with the reference answer.

[0129] The role of KL loss is to act as a regularization mechanism to enhance the alignment effect. The specific calculation steps are as follows:

[0130] 1. Get the latent representation distributions A and B of the original prompt and the generated answer.

[0131] 2. Calculate DKL(A||B) to measure the difference in their distribution in the latent space.

[0132] 3. Combine KL loss, OT loss and prompt optimization loss to optimize the generated content by minimizing the total loss to train the prompt optimization module, making it closer to the original prompt and ensuring that the generated answer is consistent with the original prompt in both semantics and structure.

[0133] Prompt optimization loss: The loss calculated by aligning the sample user query prompt and the sample reference query prompt in the latent space is used to optimize the content and expression of the prompt to make the generated results more in line with user needs.

[0134] Optionally, the prompt optimization loss can be calculated using a function loss calculation formula such as a cross entropy loss function, a Euclidean distance loss function, and a hinge loss function;

[0135] Schematically, the prompt optimization loss is calculated: Based on the alignment results of the sample user query prompts and the sample reference query prompts, the prompt optimization loss is calculated to ensure that the user query and the reference query are optimized and unified in content and expression.

[0136] Prompt optimization module parameter adjustment: Optimize the generated prompts by adjusting the parameters of the prompt optimization module to ensure that the generated content is aligned with user needs without changing the parameters of the basic large model module.

[0137] Furthermore, based on the comprehensive loss obtained by integrating the above three losses, the back propagation algorithm is executed according to the comprehensive loss to adjust the parameters of the prompt optimization module. This adjustment further optimizes the POAT module while keeping the parameters of the basic large model module unchanged to ensure the accuracy and alignment of prompt generation.

[0138] Specifically, the optimal transmission distance loss, relative entropy loss and prompt optimization loss are calculated in the latent space, and these losses are used to adjust the parameters of the prompt optimization module (POAT), so that the generated prompts and transaction processing results are more in line with user needs while keeping the parameters of the basic large model unchanged. This multi-loss fusion optimization strategy effectively improves the alignment and accuracy of the model in transaction processing and enhances the model's adaptability to complex tasks.

[0139] LLM parameter freezing: In the scheme, the parameters of LLM are frozen, which means that LLM will not be directly affected by the back-propagation gradient. Keeping LLM unchanged helps save computing resources and avoids damage to the original performance of LLM; by freezing LLM, POAT optimization can be quickly applied to adjust the generated output without the need to frequently retrain LLM.

[0140] POAT is responsible for prompt alignment: The main function of the POAT module is to generate aligned prompts and ensure that the generated prompts are aligned with the original user input query. This includes optimizing the generated answers through optimal transfer (OT) and KL regularization. After calculating the OT loss and KL loss, the back-propagation process adjusts the parameters inside POAT (such as encoder, generation module, etc.) to reduce the loss and optimize the alignment effect.

[0141] In a feasible implementation manner, the optimal transmission distance loss is calculated based on the sample original hint latent space and the sample reference query hint latent space, and the following method can be referred to:

[0142] S602: constructing an optimal transmission plan, wherein the optimal transmission plan is a movement transportation plan for moving the sample original prompt latent space to the sample reference query prompt latent space at a minimum cost;

[0143] S604: Determine a transportation cost matrix for the mobile transmission plan using a first functional formula based on the sample original prompt latent space and the sample reference query prompt latent space;

[0144] In each round of back propagation model training, the sample original prompt potential space P(P∈R M×D , M represents the number of original prompts, D represents the dimension of the embedded representation) and the sample reference query prompt latent space ( N represents the number of reference query prompts, D represents the dimension of the embedded representation) is input into a first functional formula to determine a transportation cost matrix C for the mobile transmission plan;

[0145] Wherein, the first functional formula satisfies the following formula:

[0146]

[0147] Wherein, C represents the transportation cost matrix, P represents the sample original prompt potential space, and represents the sample reference query hint latent space, T represents the mobile transportation plan for moving the sample original hint latent space to the sample reference query hint latent space; each element C in the transportation cost matrixij represents the “transfer cost” of mapping from a vector of original prompts to a vector of inference prompts, which is used to measure the similarity between the two in the latent space.

[0148] S606: Determine the optimal transmission plan using a second functional formula based on the transportation cost matrix, the sample user query prompt, and the sample reference query prompt;

[0149] The second functional formula satisfies the following formula:

[0150]

[0151] Among them, the T * represents the optimal transmission plan, the p t represents the sample user query prompt, the represents the sample reference query prompt, the diag(-C / ) represents a diagonal matrix operation function, the exp() represents a natural exponential function, and λ represents a preset regularization parameter;

[0152] In a feasible implementation manner, the determining the optimal transmission plan using a second functional formula based on the transportation cost matrix, the sample user query prompt, and the sample reference query prompt includes:

[0153] A2: Determine the current mobile transportation plan;

[0154] Schematically, the mobile transmission plan T is initialized and the edge constraint threshold ∈ is set.

[0155] For example, initialize the transmission plan parameter T=1, set the update iteration amount δp=0.01, p0=1, δp=∞.

[0156] The mobile transportation plan is a mobile transportation plan for moving the sample original prompt latent space to the sample reference query prompt latent space, with the minimum cost as the goal and the goal of constructing the optimal transmission plan;

[0157] During the model training process, the mobile transportation plan for each round is determined;

[0158] A4: Calculate the transmission distance loss of each round using the fourth functional formula based on the mobile transportation plan, the transportation cost matrix and the entropy regularization term;

[0159] The fourth functional formula satisfies the following formula:

[0160]

[0161] Wherein, OTLoss represents the transmission distance loss, λ represents a preset regularization parameter, H(T) represents an entropy regularization term, and T represents the mobile transportation plan.

[0162] Indicatively, the entropy regularization term satisfies the fifth functional formula, and the fifth functional formula satisfies the following formula:

[0163] H(T)=-∑Tlog(T)

[0164] Wherein, H(T) represents the entropy regularization term.

[0165] In this specification, in order to make the OT loss calculation more stable and efficient, an entropy regularization term is introduced and an entropy constraint is added, because the direct OT problem optimization is very time-consuming.

[0166] A6: Determine the update iteration amount δp of the mobile transportation plan;

[0167] Calculate the update iterations of the definition Used to evaluate whether the update of the current iteration is small enough to judge convergence.

[0168] A8: If the update iteration amount is greater than or equal to the edge constraint threshold, the mobile transportation plan is updated based on the transmission distance loss to obtain the updated mobile transportation plan, until the update iteration amount of the mobile transportation plan is less than the edge constraint threshold, and the optimal transmission plan is determined;

[0169] When the update iteration amount δp is greater than or equal to the edge constraint threshold ∈, it means that the current mobile transmission plan has not reached the required convergence accuracy. The model will be updated according to the gradient of the transmission distance loss OTLoss, that is, according to the gradient of the transmission distance loss OTLoss to optimize the mobile transmission plan, obtain the current mobile transportation plan, and continue to execute A2, A4, A6, A8;

[0170] The mobile transportation plan is updated based on the transmission distance loss to obtain the updated mobile transportation plan, specifically:

[0171] 1) Update sample user query prompt p t :

[0172] Using the formula Update p t , which is based on the current cost matrix C and the transmission plan Used to adjust mobile transmission plans.

[0173] 2) Update the sample reference query prompt

[0174] Using the formula renew Further adjust mobile transmission plans.

[0175] In this way, the distance between two cue embeddings in the latent space is gradually reduced.

[0176] Through continuous iteration, the large model gradually adjusts the mobile transmission plan so that the transmission distance loss is gradually reduced. At the same time, the update iteration amount δp is calculated in each round. When the update iteration amount of the mobile transportation plan is less than the edge constraint threshold, the iteration stops and the current mobile transmission plan is determined as the optimal transmission plan.

[0177] A10: Obtain the transportation cost matrix, the sample user query prompt, and the sample reference query prompt corresponding to the optimal transmission plan, and determine the optimal transmission plan using a second functional formula based on the transportation cost matrix, the sample user query prompt, and the sample reference query prompt;

[0178] The optimal transmission plan is denoted as T * represents the optimal transmission path between the generated prompt and the original prompt;

[0179] The second functional formula satisfies the following formula:

[0180]

[0181] Among them, the T * represents the optimal transmission plan, the p t represents the sample user query prompt, the represents the sample reference query prompt, the diag() represents a diagonal matrix operation function, the exp() represents a natural exponential function, and λ represents a preset regularization parameter;

[0182] diag() is usually a function used in related technologies to generate a diagonal matrix. Here, the diagonal matrix operation function diag() converts the two vectors, the generated prompt and the original prompt, into a diagonal matrix, and then performs matrix operations with other matrices. This operation is used to construct a transmission plan in the optimal transmission calculation.

[0183] exp() is usually used in related technologies to represent the natural exponential function;

[0184] S608: Determine the optimal transmission distance loss using a third function based on the transportation cost matrix and the optimal transmission plan;

[0185] The third functional formula satisfies the following formula:

[0186] Loss OT =<T*,C>

[0187] Among them, the Loss OT Indicates the optimal transmission distance loss. OT That is, the OT distance indicates the degree of alignment between the generated prompt and the original prompt in the latent space, and is an indicator to measure the similarity between the two.

[0188] Using the optimal transfer (OT) method, the OT distance is calculated to measure the alignment of the generated prompts and the original prompts in the latent space. Through iterative optimization by the Sinkhorn algorithm, the algorithm finds the optimal transfer plan for the generated prompts and the original prompts, and calculates their OT distance to achieve the alignment purpose. This process ensures that the generated prompts can accurately align the user's original input in the latent space, improving the quality and consistency of the content generated by the model.

[0189] Optionally, the prompt optimization loss is a prompt optimization cross entropy loss based on a cross entropy loss function, and the processing result loss is a processing result cross entropy loss based on a cross entropy loss function.

[0190] S5006: Calculate the processing result loss based on the predicted processing result and the reference processing result, use the processing result loss to adjust the parameters of the cross-attention module, and keep the parameters of the basic large model module unchanged.

[0191] In the model training method of this specification, inspired by the diffusion model, a cross-attention mechanism is used to insert conditional information from the prompt optimization module POAT into the basic large model module LLM. In the entire transaction processing large model, an attempt is made to efficiently insert context information with the help of a cross-attention module. The cross-attention layer of the cross-attention module can be followed by a LoRA layer as a feedforward mechanism with a jump mechanism for better learning. Therefore, based on the predicted processing result and the reference processing result, the processing result loss is calculated, and by fine-tuning the cross-attention layer using the processing result loss, we can efficiently align the output of LLM and guide the generation process, that is, the processing result loss is used to adjust the parameters of the cross-attention module, and the parameters of the basic large model module are kept unchanged.

[0192] In one or more embodiments of this specification, the process of S5002-S5006 effectively performs fine alignment of user queries and reference prompts in the latent space by calculating the optimal transmission distance loss, relative entropy loss and prompt optimization loss. This multi-loss fusion optimization strategy enables the model to better capture and understand user needs and generate highly matched responses. While keeping the parameters of the basic large model unchanged, only the parameters of the prompt optimization module are adjusted to make the generated transaction processing results more accurate and consistent, thereby greatly improving the response accuracy and alignment effect of the model in the transaction processing scenario.

[0193] See also Fig. 9 , a flowchart of a transaction processing method is provided for one or more embodiments of the present specification to train a transaction processing large model. The method can be implemented by a computer program and can be run on a transaction processing device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application. The transaction processing device can be an electronic device.

[0194] S6002: Obtaining a user query prompt input by the user;

[0195] After completing the model training process of the transaction processing large model, the transaction processing large model is then deployed to the actual transaction processing scenario;

[0196] User query prompts are user input requests or query content in actual transaction processing scenarios. For example, users may ask about specific processes or requirements, such as "How do I apply for unemployment benefits?" or "What are the application conditions for a loan?";

[0197] Exemplarily, the electronic device receives a user query prompt input by the user and uses it as input for a subsequent transaction processing model. For example, the user inputs a query prompt in the application interface, such as "How do I apply for unemployment benefits?" The electronic device uses this as input for the subsequent transaction processing process.

[0198] S6004: Input the user query prompt into the transaction processing big model, perform transaction pre-processing on the user query prompt through the transaction processing big model to obtain a reference query prompt, and perform transaction processing based on the user query prompt and the reference query prompt using a cross-attention mechanism to obtain a transaction processing result.

[0199] The transaction processing big model is an optimized and trained big model that has the ability to accurately understand and respond to user queries in transaction scenarios.

[0200] Schematically, the user query prompt is input into the transaction processing big model, and the transaction processing big model first performs transaction preprocessing to generate a reference query prompt. Specifically, the transaction processing big model can perform transaction preprocessing on the user query prompt through the prompt optimization module, extract key information or simplify the expression, and generate a reference query prompt. Then the transaction processing big model uses a cross-attention mechanism to pay attention to the information of the user query prompt and the reference query prompt at the same time to perform transaction processing to obtain the transaction processing result, so as to generate a transaction processing result that meets the requirements.

[0201] In one or more embodiments of the present specification, during the application of the transaction processing big model, a user query prompt is received and pre-processed by the transaction processing big model to obtain a reference query prompt, and a transaction processing using a cross-attention mechanism is used to generate a transaction processing result that meets the user's needs. This process ensures that the big model provides accurate and detailed transaction support in actual transaction applications, enhancing the user experience and the transaction processing capabilities of the system.

[0202] The following will be combined Fig.10 , the transaction processing device provided in this specification is introduced in detail. It should be noted that, Fig.10 The transaction processing device shown is used to execute this specification Figures 1 to 9 For the convenience of explanation, only the parts related to this specification are shown. For the specific technical details not disclosed, please refer to this specification. Figures 1 to 9 The embodiment shown.

[0203] See also Fig.10 , which shows a schematic diagram of the structure of the transaction processing device of this specification. The transaction processing device 1 can be implemented as all or part of the device through software, hardware or a combination of both. According to some embodiments, the transaction processing device 1 includes a model creation module 11 and a model training module 12, which are specifically used to:

[0204] A model creation module 11 is used to create an initial transaction processing large model based on the basic large model, and input sample user query prompts into the initial transaction processing large model for model training;

[0205] The model training module 12 is used to perform transaction preprocessing on the sample user query prompts through the initial transaction processing large model to obtain reference processing results and sample reference query prompts during the forward propagation model training process, and to perform transaction processing based on the sample user query prompts and the sample reference query prompts using a cross-attention mechanism to obtain a prediction processing result;

[0206] The model training module 12 is also used to determine the model loss based on the prediction processing results, the reference processing results, the sample user query prompts and the sample reference query prompts during the back propagation model training process, and adjust the model parameters of the initial transaction processing large model based on the model loss to obtain the transaction processing large model.

[0207] Optionally, the initial transaction processing big model includes a basic big model module, a prompt optimization module and a cross-attention module, and the model training module 12 is used to: pre-process the sample user query prompt through the basic big model module to obtain a reference processing result; perform prompt alignment processing based on the sample user query prompt and the reference query answer through the prompt optimization module to obtain a sample reference query prompt; and perform transaction processing based on the sample user query prompt and the sample reference query prompt using a cross-attention mechanism through the cross-attention module to obtain a prediction processing result.

[0208] Optionally, the model training module 12 is used to:

[0209] The prompt optimization module performs embedding processing based on the sample user query prompt and the reference query answer to obtain a sample original query prompt embedding and a sample reference answer embedding;

[0210] Performing encoding conversion processing on the sample original query prompt embedding to obtain a sample original prompt latent space, and performing encoding alignment processing on the sample reference answer embedding based on the sample original prompt latent space to obtain a sample reference answer latent space;

[0211] The sample reference answer latent space is decoded to obtain a sample reference query prompt.

[0212] Optionally, the model training module 12 is used to: determine the sample original query prompt embedding corresponding to the sample user query prompt, and determine the sample reference query prompt embedding corresponding to the sample reference query prompt; and obtain a prediction processing result by performing transaction processing using a cross-attention mechanism based on the sample original query prompt embedding and the sample reference query prompt embedding through the cross-attention module.

[0213] Optionally, the initial transaction processing large model includes a basic large model module, a prompt optimization module and a cross-attention module, and the model training module 12 is used to: obtain the sample original prompt latent space corresponding to the sample user query prompt, and the sample reference query prompt latent space corresponding to the sample reference query prompt; calculate the optimal transmission distance loss and relative entropy loss based on the sample original prompt latent space and the sample reference query prompt latent space, calculate the prompt optimization loss based on the sample user query prompt and the sample reference query prompt, adjust the parameters of the prompt optimization module based on the optimal transmission distance loss, relative entropy loss and the prompt optimization loss, and keep the parameters of the basic large model module unchanged; calculate the processing result loss based on the predicted processing result and the reference processing result, use the processing result loss to adjust the parameters of the cross-attention module, and keep the parameters of the basic large model module unchanged.

[0214] Optionally, the model training module 12 is used to: construct an optimal transmission plan, wherein the optimal transmission plan is a mobile transportation plan for moving the sample original prompt latent space to the sample reference query prompt latent space at a minimum cost;

[0215] Determine a transportation cost matrix for the mobile transmission plan using a first functional formula based on the sample original prompt latent space and the sample reference query prompt latent space;

[0216] Determine the optimal transport plan using a second functional formula based on the transport cost matrix, the sample user query prompt and the sample reference query prompt;

[0217] Determine the optimal transmission distance loss using a third functional formula based on the transportation cost matrix and the optimal transmission plan;

[0218] The first functional formula satisfies the following formula:

[0219]

[0220] Wherein, C represents the transportation cost matrix, P represents the sample original prompt potential space, and represents the sample reference query hint latent space, T represents a moving transportation plan for moving the sample original hint latent space to the sample reference query hint latent space;

[0221] The second functional formula satisfies the following formula:

[0222]

[0223] Among them, the T * represents the optimal transmission plan, the pt represents the sample user query prompt, the represents the sample reference query prompt, the diag() represents a diagonal matrix operation function, the exp() represents a natural exponential function, and λ represents a preset regularization parameter;

[0224] The third functional formula satisfies the following formula:

[0225] Loss OT =<T*,C>

[0226] Among them, the Loss OT represents the optimal transmission distance loss.

[0227] Optionally, the model training module 12 is used to: determine the current mobile transportation plan;

[0228] Calculate the transmission distance loss of each round using a fourth functional formula based on the mobile transportation plan, the transportation cost matrix and the entropy regularization term;

[0229] Determining an update iteration amount of the mobile transportation plan;

[0230] If the update iteration amount is greater than or equal to the edge constraint threshold, the mobile transportation plan is updated based on the transmission distance loss to obtain the updated mobile transportation plan, until the update iteration amount of the mobile transportation plan is less than the edge constraint threshold, and the optimal transmission plan is determined;

[0231] Obtaining the transportation cost matrix, the sample user query prompt, and the sample reference query prompt corresponding to the optimal transmission plan, and determining the optimal transmission plan using a second functional formula based on the transportation cost matrix, the sample user query prompt, and the sample reference query prompt;

[0232] The fourth functional formula satisfies the following formula:

[0233]

[0234] Wherein, OTLoss represents the transmission distance loss, λ represents a preset regularization parameter, H(T) represents an entropy regularization term, and T represents the mobile transportation plan.

[0235] Optionally, the model training module 12 is used to: the prompt optimization loss is a prompt optimization cross entropy loss based on a cross entropy loss function, and the processing result loss is a processing result cross entropy loss based on a cross entropy loss function.

[0236] See also Fig.11, which shows a schematic diagram of the structure of the transaction processing device of this specification. The transaction processing device 2 can be implemented as all or part of the device through software, hardware or a combination of both. According to some embodiments, the transaction processing device 2 includes a prompt acquisition module 21 and a transaction processing module 22, which are specifically used to:

[0237] A prompt acquisition module 21 is used to acquire a user query prompt input by a user;

[0238] The transaction processing module 22 is used to input the user query prompt into the transaction processing big model, perform transaction pre-processing on the user query prompt through the transaction processing big model to obtain a reference query prompt, and perform transaction processing based on the user query prompt and the reference query prompt using a cross-attention mechanism to obtain a transaction processing result.

[0239] It should be noted that the transaction processing device provided in the above embodiment only uses the division of the above functional modules as an example when executing the transaction processing method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the transaction processing device provided in the above embodiment and the transaction processing method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0240] The above serial numbers in this specification are for description only and do not represent the advantages or disadvantages of the embodiments.

[0241] The present specification also provides a computer storage medium, which can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figures 1 to 9 The transaction processing method of the embodiment shown in the figure can be found in the specific execution process. Figures 1 to 9 The specific description of the illustrated embodiment will not be repeated here.

[0242] The present specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figures 1 to 9 The transaction processing method of the embodiment shown in the figure can be found in the specific execution process. Figures 1 to 9 The specific description of the illustrated embodiment will not be repeated here.

[0243] Please refer to Fig.12, is a block diagram of a structure of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 1010, a memory 1020, an input device 1030, an output device 1040, and a bus 1050. The processor 1010, the memory 1020, the input device 1030, and the output device 1040 may be connected via a bus 1050.

[0244] The processor 1010 may include one or more processing cores. The processor 1010 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1020, and calling data stored in the memory 1020. Optionally, the processor 1010 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1010 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1010, but may be implemented separately through a communication chip.

[0245] The memory 1020 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1020 includes a non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, codes, code sets, or instruction sets.

[0246] The input device 1030 is used to receive input instructions or data, and the input device 1030 includes but is not limited to a keyboard, a mouse, a camera, a microphone or a touch device. The output device 1040 is used to output instructions or data, and the output device 1040 includes but is not limited to a display device and a speaker. In the embodiment of this specification, the input device 1030 can be a temperature sensor for obtaining the operating temperature of the electronic device. The output device 1040 can be a speaker for outputting audio signals.

[0247] In addition, those skilled in the art will appreciate that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WIFI) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0248] In the embodiments of this specification, the execution subject of each step may be the electronic device described above. Optionally, the execution subject of each step is the operating system of the electronic device. The operating system may be an Android system, an IOS system, or other operating systems, which is not limited in the embodiments of this specification.

[0249] exist Fig.12 In the electronic device, the processor 1010 can be used to call the program stored in the memory 1020 and execute it to implement the transaction processing method as described in the various method embodiments of this specification.

[0250] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0251] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the user query prompts and transaction processing results involved in this specification are all obtained with full authorization.

[0252] The above disclosure is only the preferred embodiment of this specification, which certainly cannot be used to limit the scope of rights of this specification. Therefore, equivalent changes made according to the claims of this specification are still within the scope covered by this specification.

Claims

1. A transaction processing method, the method comprising: Create an initial transaction processing model based on the basic model, and input sample user query prompts into the initial transaction processing model for model training; In the forward propagation model training process, the sample user query prompt is subjected to transaction preprocessing by the initial transaction processing large model to obtain a reference processing result and a sample reference query prompt, and a cross-attention mechanism is used to perform transaction processing based on the sample user query prompt and the sample reference query prompt to obtain a prediction processing result; During the back-propagation model training process, the model loss is determined based on the predicted processing results, the reference processing results, the sample user query prompts, and the sample reference query prompts, and the model parameters of the initial transaction processing large model are adjusted based on the model loss to obtain the transaction processing large model.

2. The method according to claim 1, wherein the initial transaction processing big model comprises a basic big model module, a prompt optimization module and a cross attention module, The step of performing transaction preprocessing on the sample user query prompt by using the initial transaction processing large model to obtain a reference processing result and a sample reference query prompt, and performing transaction processing based on the sample user query prompt and the sample reference query prompt using a cross-attention mechanism to obtain a prediction processing result includes: Preprocessing the sample user query prompts through the basic large model module to obtain a reference processing result; The prompt optimization module performs prompt alignment processing based on the sample user query prompt and the reference query answer to obtain a sample reference query prompt; The cross-attention module uses a cross-attention mechanism to perform transaction processing based on the sample user query prompt and the sample reference query prompt to obtain a prediction processing result.

3. The method according to claim 2, wherein the prompt optimization module performs prompt alignment processing based on the sample user query prompt and the reference query answer to obtain the sample reference query prompt, comprising: The prompt optimization module performs embedding processing based on the sample user query prompt and the reference query answer to obtain a sample original query prompt embedding and a sample reference answer embedding; Performing encoding conversion processing on the sample original query prompt embedding to obtain a sample original prompt latent space, and performing encoding alignment processing on the sample reference answer embedding based on the sample original prompt latent space to obtain a sample reference answer latent space; The sample reference answer latent space is decoded to obtain a sample reference query prompt.

4. The method according to claim 2, wherein the cross-attention module uses a cross-attention mechanism to perform transaction processing based on the sample user query prompt and the sample reference query prompt to obtain a prediction processing result, comprising: Determining a sample original query prompt embedding corresponding to the sample user query prompt, and determining a sample reference query prompt embedding corresponding to the sample reference query prompt; The cross-attention module uses a cross-attention mechanism to perform transaction processing based on the sample original query prompt embedding and the sample reference query prompt embedding to obtain a prediction processing result.

5. The method according to claim 1, wherein the initial transaction processing big model comprises a basic big model module, a prompt optimization module and a cross attention module, The determining of the model loss based on the prediction processing result, the reference processing result, the sample user query prompt and the sample reference query prompt, and adjusting the model parameters of the initial transaction processing large model based on the model loss, comprises: Acquire a sample original prompt latent space corresponding to the sample user query prompt and a sample reference query prompt latent space corresponding to the sample reference query prompt; Based on the sample original prompt latent space and the sample reference query prompt latent space, calculate the optimal transmission distance loss and the relative entropy loss, calculate the prompt optimization loss based on the sample user query prompt and the sample reference query prompt, adjust the parameters of the prompt optimization module based on the optimal transmission distance loss, the relative entropy loss and the prompt optimization loss, and keep the parameters of the basic large model module unchanged; The processing result loss is calculated based on the predicted processing result and the reference processing result, and the parameters of the cross-attention module are adjusted using the processing result loss, while the parameters of the basic large model module are kept unchanged.

6. The method according to claim 5, wherein the step of calculating the optimal transmission distance loss based on the sample original hint latent space and the sample reference query hint latent space comprises: Constructing an optimal transport plan, wherein the optimal transport plan is a transport plan for moving the sample original prompt latent space to the sample reference query prompt latent space with minimum cost; Determine a transportation cost matrix for the mobile transmission plan using a first functional formula based on the sample original prompt latent space and the sample reference query prompt latent space; Determine the optimal transport plan using a second functional formula based on the transport cost matrix, the sample user query prompt and the sample reference query prompt; Determine the optimal transmission distance loss using a third functional formula based on the transportation cost matrix and the optimal transmission plan; The first functional formula satisfies the following formula: Wherein, C represents the transportation cost matrix, P represents the sample original prompt potential space, and represents the sample reference query hint latent space, T represents a moving transportation plan for moving the sample original hint latent space to the sample reference query hint latent space; The second functional formula satisfies the following formula: Among them, the T * represents the optimal transmission plan, the p t represents the sample user query prompt, the represents the sample reference query prompt, the diag() represents a diagonal matrix operation function, the exp() represents a natural exponential function, and λ represents a preset regularization parameter; The third functional formula satisfies the following formula: Loss OT =<T*,C> Among them, the Loss OT represents the optimal transmission distance loss.

7. The method according to claim 6, wherein the determining the optimal transport plan using a second functional formula based on the transport cost matrix, the sample user query prompt and the sample reference query prompt comprises: determining a current plan for said mobile transportation; Calculate the transmission distance loss of each round using a fourth functional formula based on the mobile transportation plan, the transportation cost matrix and the entropy regularization term; Determining an update iteration amount of the mobile transportation plan; If the update iteration amount is greater than or equal to the edge constraint threshold, the mobile transportation plan is updated based on the transmission distance loss to obtain the updated mobile transportation plan, until the update iteration amount of the mobile transportation plan is less than the edge constraint threshold, and the optimal transmission plan is determined; Obtaining the transportation cost matrix, the sample user query prompt, and the sample reference query prompt corresponding to the optimal transmission plan, and determining the optimal transmission plan using a second functional formula based on the transportation cost matrix, the sample user query prompt, and the sample reference query prompt; The fourth functional formula satisfies the following formula: Among them, OTLoss represents the transmission distance loss, λ represents a preset regularization parameter, H(T) represents an entropy regularization term, and T represents the mobile transportation plan.

8. According to the method of claim 5, the prompt optimization loss is a prompt optimization cross entropy loss based on a cross entropy loss function, and the processing result loss is a processing result cross entropy loss based on a cross entropy loss function.

9. A transaction processing method, the method comprising: Get the user query prompt entered by the user; The user query prompt is input into a transaction processing big model, the user query prompt is subjected to transaction preprocessing by the transaction processing big model to obtain a reference query prompt, and transaction processing is performed based on the user query prompt and the reference query prompt using a cross-attention mechanism to obtain a transaction processing result.

10. A transaction processing device, comprising: A model creation module is used to create an initial transaction processing model based on the basic model, and input sample user query prompts into the initial transaction processing model for model training; A model training module, used for performing transaction preprocessing on the sample user query prompts by using the initial transaction processing large model to obtain reference processing results and sample reference query prompts during the forward propagation model training process, and performing transaction processing based on the sample user query prompts and the sample reference query prompts by using a cross-attention mechanism to obtain a prediction processing result; The model training module is also used to determine the model loss based on the prediction processing results, the reference processing results, the sample user query prompts and the sample reference query prompts during the back propagation model training process, and adjust the model parameters of the initial transaction processing large model based on the model loss to obtain the transaction processing large model.

11. A transaction processing device, comprising: A prompt acquisition module is used to obtain the user query prompt input by the user; A transaction processing module is used to input the user query prompt into a transaction processing large model, perform transaction preprocessing on the user query prompt through the transaction processing large model to obtain a reference query prompt, and perform transaction processing based on the user query prompt and the reference query prompt using a cross-attention mechanism to obtain a transaction processing result.

12. A computer storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 8 or 9.

13. A computer program product, the computer program product storing at least one instruction, wherein the at least one instruction is loaded by a processor and executes the method steps according to any one of claims 1 to 8 or 9.

14. An electronic device, comprising: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 8 or 9.

Images