Data processing method, device and equipment

Abstract

The embodiment of the invention provides a data processing method, device and equipment, and the method comprises the steps: obtaining first training data which comprises historical business data, a transaction risk detection result corresponding to the historical business data, and an illusion detection label corresponding to the transaction risk detection result, determining a first illusion detection result based on the historical business data and the transaction risk detection result by using a preset big language model, and determining a second illusion detection result based on preset prompt information, the historical business data, the transaction risk detection result and a corresponding illusion detection label by using the preset big language model, and determining a target illusion tag corresponding to the transaction risk detection result based on the illusion detection tag, the first illusion detection result and the second illusion detection result, and training a target model based on the transaction risk detection result and the corresponding target illusion tag.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a data processing method, device and equipment. BACKGROUND

[0002] With the continuous improvement of the digitization process of financial business, illegal transactions and other risk behaviors are increasingly showing the characteristics of concealment, intelligence and cross-platform. How to detect the risk of transaction users to protect user privacy and ensure transaction and data security has become the focus of various industries.

[0003] A model constructed based on a deep learning algorithm such as a large language model can be used to detect the risk of transaction users. However, due to the inherent mechanism of the large language model and the limitation of the training data, factual errors and other model hallucination may occur, resulting in low accuracy of risk detection. Therefore, a hallucination detection scheme for more reliable transaction risk detection results is needed to improve the accuracy of transaction risk detection. SUMMARY

[0004] The purpose of the embodiments of the present specification is to provide a hallucination detection scheme for more reliable transaction risk detection results to improve the accuracy of transaction risk detection.

[0005] In order to achieve the above technical solutions, the embodiments of the present specification are implemented as follows: The data processing method provided by the embodiments of the present specification comprises: obtaining first training data, wherein the first training data comprises historical business data, a transaction risk detection result corresponding to the historical business data, and a hallucination detection label corresponding to the transaction risk detection result; determining a first hallucination detection result based on the historical business data and the transaction risk detection result by using a preset large language model; determining a second hallucination detection result based on preset prompt information, the historical business data, the transaction risk detection result and the corresponding hallucination detection label by using the preset large language model, wherein the preset prompt information is used to constrain the preset large language model to detect whether the transaction risk detection result has hallucination under the given hallucination detection label; determining a target hallucination label corresponding to the transaction risk detection result based on the hallucination detection label, the first hallucination detection result and the second hallucination detection result; and training a target model based on the transaction risk detection result and the corresponding target hallucination label, so as to detect whether the transaction risk detection result output by a preset model has hallucination based on the trained target model.

[0006] The embodiment of the specification provides a data processing device, the device comprises: a first acquisition module for acquiring first training data, the first training data comprising historical business data, a transaction risk detection result corresponding to the historical business data, and an illusion detection label corresponding to the transaction risk detection result; a first detection module for determining a first illusion detection result based on the historical business data and the transaction risk detection result by using a preset large language model; a second detection module for determining a second illusion detection result based on preset prompt information, the historical business data, the transaction risk detection result, and the corresponding illusion detection label by using the preset large language model, the preset prompt information being used to constrain the preset large language model to detect whether there is an illusion in the transaction risk detection result given the illusion detection label; a label determination module for determining a target illusion label corresponding to the transaction risk detection result based on the illusion detection label, the first illusion detection result, and the second illusion detection result; and a model training module for training a target model based on the transaction risk detection result and the corresponding target illusion label, so as to detect whether there is an illusion in a transaction risk detection result output by a preset model based on the trained target model.

[0007] The embodiment of the specification provides a data processing device, the device comprises: a processor; and a memory arranged to store computer executable instructions, the executable instructions, when executed, causing the processor to: acquire first training data, the first training data comprising historical business data, a transaction risk detection result corresponding to the historical business data, and an illusion detection label corresponding to the transaction risk detection result; determine a first illusion detection result based on the historical business data and the transaction risk detection result by using a preset large language model; determine a second illusion detection result based on preset prompt information, the historical business data, the transaction risk detection result, and the corresponding illusion detection label by using the preset large language model, the preset prompt information being used to constrain the preset large language model to detect whether there is an illusion in the transaction risk detection result given the illusion detection label; determine a target illusion label corresponding to the transaction risk detection result based on the illusion detection label, the first illusion detection result, and the second illusion detection result; and train a target model based on the transaction risk detection result and the corresponding target illusion label, so as to detect whether there is an illusion in a transaction risk detection result output by a preset model based on the trained target model.

[0008] This specification also provides a storage medium for storing computer-executable instructions. When executed by a processor, these instructions implement the following process: acquiring first training data, including historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results; using a preset large language model, determining a first illusion detection result based on the historical business data and the transaction risk detection results; using the preset large language model, determining a second illusion detection result based on preset prompt information, the historical business data, the transaction risk detection results, and the corresponding illusion detection labels, wherein the preset prompt information constrains the preset large language model to detect whether the transaction risk detection results are illusory given the illusion detection labels; determining a target illusion label corresponding to the transaction risk detection results based on the illusion detection labels, the first illusion detection result, and the second illusion detection result; and training a target model based on the transaction risk detection results and the corresponding target illusion labels to detect whether the transaction risk detection results output by the preset model are illusory based on the trained target model.

[0009] This specification also provides a computer program product, including a computer program that, when executed by a processor, implements the following process: acquiring first training data, the first training data including historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results; using a preset large language model, determining a first illusion detection result based on the historical business data and the transaction risk detection results; using the preset large language model, determining a second illusion detection result based on preset prompt information, the historical business data, the transaction risk detection results, and the corresponding illusion detection labels, the preset prompt information constraining the preset large language model to detect whether the transaction risk detection results are illusory given the illusion detection labels; determining a target illusion label corresponding to the transaction risk detection results based on the illusion detection labels, the first illusion detection results, and the second illusion detection results; and training a target model based on the transaction risk detection results and the corresponding target illusion labels, so as to detect whether the transaction risk detection results output by the preset model are illusory based on the trained target model. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram illustrating the implementation environment of one data processing method described in this specification. Figure 2 This is a schematic diagram illustrating the processing procedure of one data processing method described in this specification; Figure 3 This is a schematic diagram of a transaction risk detection process described in this manual; Figure 4 This is a schematic diagram of the data processing process of a dual-model collaborative system described in this specification; Figure 5 This is a schematic diagram of a reinforcement learning training process described in this specification; Figure 6 This is a schematic diagram illustrating the process of determining a target illusion tag as described in this specification; Figure 7 This is a schematic diagram illustrating the construction process of a training sample pool as described in this specification; Figure 8 This is a schematic diagram of a preprocessing procedure described in this specification; Figure 9 This is a schematic diagram of a prompt message in this instruction manual; Figure 10 This is a schematic diagram of another transaction risk detection process described in this manual; Figure 11 This is a schematic diagram of a detection system described in this specification; Figure 12 This is a schematic diagram of a data processing device described in this specification; Figure 13 This is a schematic diagram of a data processing device described in this specification. Detailed Implementation

[0011] This specification provides a data processing method, apparatus, and device through its embodiments.

[0012] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0013] This specification provides a more reliable illusion detection scheme for transaction risk detection results to improve the accuracy of transaction risk detection. In practical applications, illusion detection can be performed on transaction risk detection results based on rule-based strategies. For example, it is possible to identify potential illusion problems in transaction risk detection results on a standard evaluation set. However, since rule-based strategies rely on manually designed rules, illusion detection is singular and highly customized, with poor generalizability. Alternatively, illusion detection can be performed on transaction risk detection results through manual sampling. For example, it is possible to manually sample and review the transaction risk detection results output by the model to check for illusions and manually annotate the areas where illusion problems exist. However, manual sampling is time-consuming, has low review efficiency, and manual judgment has problems such as inconsistent standards and judgment errors, which can easily lead to false detections or missed illusion problems. To this end, this specification provides various technical solutions to improve the efficiency and accuracy of contract revision. These solutions utilize a pre-set large language model, employing both forward modeling (i.e., performing illusion detection without a given illusion detection label) and backward modeling (i.e., performing illusion detection with a given illusion detection label) to detect whether illusions exist in the transaction risk detection results. This ensures the accuracy of the target illusion label determined based on the illusion detection label, the first illusion detection result, and the second illusion detection result. Furthermore, by using the transaction risk detection results and the corresponding target illusion label, the illusion detection effect of the target model can be improved, thereby enhancing the accuracy of transaction risk detection based on the trained target model. Specific processing details can be found in the following embodiments.

[0014] The data processing methods described in one or more embodiments of this specification are applicable to the data processing implementation environment, such as... Figure 1 As shown, the implementation environment includes at least: Client 100 and server 200. Furthermore, server 200 can be configured with various network models and algorithms, among which: Client 100 can run on terminal devices, which can be mobile phones, personal computers, tablets, e-book readers, wearable devices, devices that interact with information based on AR (Augmented Reality) and VR (Virtual Reality), and laptop computers, etc. Client 100 can be installed on terminal devices, and Client 100 can be an application, a browser, or a subroutine embedded in an application, etc.

[0015] Server 200 can run on a server, which can be one or more servers, a server cluster consisting of several servers, or a cloud server on a cloud computing platform. Server 200 can be installed on the server. Server 200 can be an application or a subroutine embedded in an application. Various network models and algorithms can be integrated into server 200, or server 200 can call one or more of various network models and algorithms to perform corresponding operations.

[0016] In addition, it may include a database 300, which may be located in the server on which the server 200 runs, or outside the server on which the server 200 runs. The database 300 may store sample data for training the model, such as first training data and second training data, as well as model parameters of the model, such as the target model and the preset model after training.

[0017] In this implementation environment, server 200 can obtain first training data, which may include historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results. Using a pre-set large language model, based on the historical business data and transaction risk detection results, a first illusion detection result is determined. Using the same pre-set large language model, based on pre-set prompts, historical business data, transaction risk detection results, and corresponding illusion detection labels, a second illusion detection result is determined. The pre-set prompts can be used to constrain the pre-set large language model to determine the transaction risk based on given illusion detection labels. The system detects whether the risk detection result is an illusion. Based on the illusion detection label, the first illusion detection result, and the second illusion detection result, it determines the target illusion label corresponding to the transaction risk detection result. Based on the transaction risk detection result and the corresponding target illusion label, it trains the target model. The server 200 uses the preset model to perform transaction risk detection on the relevant data of the object to be detected sent by the client 100. Then, the server 200 can detect whether the transaction risk detection result output by the preset model is an illusion based on the trained target model, and return the transaction risk detection result and / or the corresponding illusion detection result to the client 100.

[0018] like Figure 2 As shown in the embodiments of this specification, a data processing method is provided. The execution subject of this method can be a server, which can be a single independent server or a server cluster composed of multiple servers. The server can be a backend server for businesses such as financial services or online shopping services, or a backend server for a certain application. The method may specifically include the following steps: In step S202, the first training data is obtained.

[0019] The first training data may include historical business data, corresponding transaction risk detection results, and illusion detection labels corresponding to the transaction risk detection results. Historical business data may be relevant data on the transaction behavior of historical trading entities, such as interaction data, behavioral time-series data, and transaction relationship diagram structure data related to the transaction behavior of historical trading entities. Transaction risk detection results may be judgments based on transaction feedback data or human experience. These results may include a conclusion regarding the existence of transaction risk and the types of transaction risks involved. Transaction risk types may include illegal financial activities (which may involve a series of regulations, laws, and procedures to prevent users from laundering their illegal gains through the financial system; these regulations require financial institutions and other regulated entities to...). Entities take measures to identify, monitor and prevent suspicious financial activities, fraud, illegal operations, etc. For example, the transaction risk detection result corresponding to the first training data can be that there is a transaction risk and the transaction risk type is illegal operation, or the transaction risk detection result corresponding to the first training data can be that there is no transaction risk. The illusion detection label corresponding to the transaction risk detection result can be used to characterize whether there is an illusion in the transaction risk detection result and the type of illusion involved. The illusion type can include factual illusion, faithful illusion and logical illusion, etc. Factual illusion can be used to characterize the model output result (i.e. transaction risk detection result) as having problems such as feature referencing errors and feature fabrication. Faithful illusion can be used to characterize the model output result as inconsistent with the instruction. Logical illusion type can be used to characterize the model output result as having problems such as logical contradiction.

[0020] In implementation, the server can perform transaction risk detection on the business data in the generation link based on a preset quality inspection cycle, and identify the business data with transaction risk detection as historical business data. Alternatively, the server can also identify historical business data based on Suspicious Transaction Reports (STRs) and / or Enhanced Due Diligence (EDDs). There are various methods for obtaining historical business data, and this specification does not specifically limit them in the embodiments.

[0021] After acquiring historical business data, the server can use models built based on deep learning algorithms, such as preset big language, to perform transaction risk detection on the historical business data, and send the historical business data and the corresponding transaction risk detection results to the preset reviewer. Based on the feedback from the preset reviewer, the server can determine the illusion detection label corresponding to the transaction risk detection result.

[0022] In step S204, the first illusion detection result is determined by using a preset large language model based on historical business data and transaction risk detection results.

[0023] In implementation, the server can input historical business data and transaction risk detection results into a preset large language model to detect whether there is an illusion in the transaction risk detection results and obtain the first illusion detection result.

[0024] In step S206, a second illusion detection result is determined using a preset large language model based on preset prompt information, historical business data, transaction risk detection results, and corresponding illusion detection labels.

[0025] Among them, the preset prompt information can be used to constrain the preset large language model to detect whether there is a hallucination in the transaction risk detection result given the hallucination detection label.

[0026] In implementation, the server can construct preset prompts based on illusion detection labels. For example, the preset prompts could be "The transaction risk detection result may have problems such as factual illusion, fidelity illusion, and logical illusion. Please determine whether the transaction risk detection result has the above-mentioned illusion problems." In this way, the preset prompts can constrain the preset large language model to perform illusion detection by reverse reasoning and obtain the second illusion detection result.

[0027] Alternatively, the server can classify the first training data according to the hallucination type corresponding to the hallucination detection label, and configure corresponding preset prompts for the first training data of the same hallucination type to improve the model detection effect in a targeted manner.

[0028] In step S208, the target illusion label corresponding to the transaction risk detection result is determined based on the illusion detection label, the first illusion detection result, and the second illusion detection result.

[0029] In practice, the server can send the illusion detection label, the first illusion detection result, and the second illusion detection result to the preset reviewer, and determine the target illusion label corresponding to the transaction risk detection result based on the feedback from the preset reviewer.

[0030] Furthermore, the hallucination detection label, the first hallucination detection result, and the second hallucination detection result may include the detection conclusion of whether a hallucination exists, as well as the type of hallucination involved. The server can perform matching processing on the detection conclusions and hallucination types contained in these three documents to determine the target hallucination label based on the matching results. For example, if the detection conclusions contained in the three documents are consistent, the hallucination detection label can be determined as the target hallucination label.

[0031] Alternatively, if the detection conclusions contained in the three are inconsistent, the target illusion label corresponding to the transaction risk detection result can be determined by further considering one or more of the following: the weights of the three, the types of illusions involved, and business data involving illusion issues in historical business data.

[0032] For example, the server can use a pre-trained label determination model to determine the target hallucination label corresponding to the transaction risk detection result based on the hallucination detection label, the weights corresponding to the first hallucination detection result and the second hallucination detection result, the types of hallucination involved, and the business data involving hallucination issues in historical business data. The label determination model can be a model built based on a preset machine learning algorithm.

[0033] Furthermore, the method for determining the target illusion label described above is an optional and feasible method. In practical application scenarios, there can be a variety of different methods. Different methods can be selected according to different practical application scenarios. This specification does not make specific limitations on this.

[0034] In step S210, the target model is trained based on the transaction risk detection results and the corresponding target illusion labels, so as to detect whether there is an illusion in the transaction risk detection results output by the preset model based on the trained target model.

[0035] The target model can be any model capable of hallucination detection. For example, the target model can be a model built based on a preset deep learning algorithm, such as a Large Language Model (LLM), a Multi-model Large Language Model (MLLM), or a model built based on a neural network algorithm. A Large Language Model can be an artificial intelligence model trained on massive amounts of text data, possessing the ability to understand, generate, and interact with natural language. It can handle complex language tasks and output results that conform to human language habits. A Multi-model Large Language Model is an artificial intelligence model that can simultaneously understand and generate multiple types of information (such as text, images, audio, video, etc.). It extends the understanding of non-textual modalities on the basis of the Large Language Model, thereby realizing cross-modal interaction and reasoning.

[0036] In practice, the target model can be trained in a supervised manner using the transaction risk detection results and the corresponding target illusion labels to obtain the trained target model.

[0037] In addition, to improve the accuracy of determining the target illusion label, the number of parameters of the preset large language model can be greater than the preset parameter level threshold. Similarly, to improve the training efficiency of the target model and the detection efficiency of the illusion detection using the trained target model, the number of parameters of the target model can be less than the preset parameter level threshold. That is, the target illusion label corresponding to the transaction risk detection result can be determined by using the forward and backward inference results of the large parameter model (i.e., the preset large language model). Then, the lightweight illusion detection model (i.e., the target model) can be quickly trained using the transaction risk detection result and the corresponding target illusion label to improve the efficiency and accuracy of illusion detection.

[0038] This specification provides a data processing method that can acquire first training data, which may include historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results. Using a preset large language model, a first illusion detection result is determined based on the historical business data and the transaction risk detection results. Using the preset large language model, a second illusion detection result is determined based on preset prompt information, historical business data, the transaction risk detection results, and the corresponding illusion detection labels. The preset prompt information can be used to constrain the preset large language model to detect whether the transaction risk detection results are illusory given the illusion detection labels. Based on the illusion detection labels, the first illusion detection result, and the second illusion detection result, a target illusion label corresponding to the transaction risk detection result is determined. Based on the transaction risk detection results and the corresponding target illusion label, a target model is trained to detect whether the transaction risk detection results output by the preset model are illusory. In this way, by using a pre-set large language model, and through model forward inference (i.e., performing hallucination detection without giving hallucination detection labels) and model backward inference (i.e., performing hallucination detection with given hallucination detection labels), the presence of hallucinations in the transaction risk detection results can be detected. This ensures the accuracy of the target hallucination label determined based on the hallucination detection label, the first hallucination detection result, and the second hallucination detection result. Furthermore, by using the transaction risk detection results and the corresponding target hallucination label, the hallucination detection effect of the target model can be improved, thereby improving the accuracy of transaction risk detection based on the trained target model.

[0039] In practical applications, the trained target model can be used to filter sample data, and then the filtered sample data can be used to detect transaction risks of trading objects. There are various methods for filtering sample data; the following provides one optional method, such as... Figure 3 As shown, the specific process may include the following steps S302 to S310.

[0040] In step S302, the second training data is obtained.

[0041] The second training data can be high-quality training sample data constructed from highly accurate (i.e., transaction risk detection accuracy is higher than the preset accuracy threshold) transaction business data determined based on business experience, and highly difficult (i.e., transaction risk detection difficulty is higher than the preset difficulty threshold) transaction business data obtained through manual review.

[0042] In step S304, the first detection model is used to detect transaction risks on the second training data to obtain the first detection result.

[0043] The first detection model is a model obtained by training a model based on a preset machine learning algorithm using historical data. For example, the first detection model can be a model obtained by training a model based on a neural network algorithm, a preset large language model, or a preset multimodal large language model using historical transaction data. In addition, to improve the accuracy of transaction risk detection, the number of parameters in the first detection model can be greater than the preset parameter level threshold.

[0044] In implementation, the server can input the second training data into the first detection model to obtain the first detection result. The first detection result may include a detection conclusion on whether there is a transaction risk, as well as the corresponding inference chain.

[0045] In step S306, the trained target model is used to detect whether the first detection result is a hallucination, and a third hallucination detection result is obtained.

[0046] The third hallucination detection result can include the detection conclusion of whether hallucination exists, as well as the corresponding hallucination type.

[0047] In step S308, based on the third hallucination detection result, the second training data is filtered to obtain the third training data.

[0048] In practice, the server can determine the second training data corresponding to the first detection result where no hallucination exists as the third training data based on the detection conclusion in the third hallucination detection result.

[0049] Alternatively, the server can filter the second training data based on the third illusion detection results, according to the model training requirements of the application scenario to which the second detection model belongs, to obtain the third training data.

[0050] For example, assuming that the model training requirements of the application scenario to which the second detection model belongs include factual illusion requirements and faithful illusion requirements, then the server can determine the second training data corresponding to the first detection results that do not contain illusions and those that contain illusions but whose illusion type is logical illusion as the third training data.

[0051] In step S310, the second detection model is trained based on the third training data, so as to detect whether there is a transaction risk in the transaction object to be detected based on the trained second detection model.

[0052] The number of parameters in the second detection model can be smaller than that in the first detection model.

[0053] In implementation, the first detection model can be a teacher model, and the second detection model can be a student model. That is, based on the first and second detection models, a model like... Figure 4 The dual-model collaborative system shown has a first detection model (i.e., the teacher model) that can be used to generate professional knowledge reasoning results (i.e., the first detection results). The illusion detection model (i.e., the target model) trained on illusion labeled samples (i.e., transaction risk detection results and corresponding target illusion labels) ensures that the output is suitable as the third training data for the second detection model.

[0054] In addition, such as Figure 4 As shown, the output instructions of the third training data can be continuously improved through an iterative reflection mechanism, which can select high-quality samples for training the student model. Specifically, if the second training data is determined to contain hallucinations based on the third hallucination detection result, the output result of the hallucination detection model (i.e., the third hallucination detection result) can be fed back to the teacher model for reflection and retry. If the number of reflections exceeds the set number, then the second training data that still contains hallucinations can be sent to a human for review, and the second training data after review will be determined as the third training data.

[0055] The student model was trained using the third training data obtained by distilling the second training data based on the hallucination detection model. The trained student model outperformed the student model trained on the undistilled second training data in terms of precision and recall.

[0056] In practical applications, there are various ways to train the second detection model based on the third training data in step S310 above. The following is an optional processing method, which may include the following steps A1 to A4.

[0057] In step A1, the transaction risk label corresponding to the third training data is obtained.

[0058] In step A2, the second detection model is used to detect transaction risks on the third training data to obtain the second detection result.

[0059] In practice, the server can input the third training data into the second detection model to use the second detection model to detect transaction risks and obtain the second detection result corresponding to the third training data.

[0060] In step A3, the trained target model is used to detect whether the second detection result is a hallucination, thus obtaining the fourth hallucination detection result.

[0061] In practice, the server can input the second detection result (and the third training data) into the trained target model to perform hallucination detection and obtain the fourth hallucination detection result corresponding to the second detection result.

[0062] In step A4, the second detection model is trained based on the fourth illusion detection result, the second detection result, and the transaction risk label.

[0063] In implementation, the server can perform supervised training on the second detection model based on the fourth illusion detection results, the second detection results, and the transaction risk label. For example, the server can determine the loss value based on a preset loss function, the fourth illusion detection results, the second detection results, and the transaction risk label, and determine whether the second detection model has converged based on the loss value. If it is determined that the second detection model has not converged, the server can continue to train the second detection model based on the fourth illusion detection results, the second detection results, and the transaction risk label until the second detection model converges, thus obtaining the trained second detection model.

[0064] Alternatively, the server can train the second detection model using reinforcement learning based on the fourth illusion detection result, the second detection result, and the transaction risk label. For example, the fourth illusion detection result may include the detection conclusion of whether an illusion exists, the type of illusion involved, and the data where an illusion exists. Based on the fourth illusion detection result, the server can determine the proportion of data where an illusion does not exist and / or where an illusion exists in the second detection result, and determine the first reward information. For example, the server can determine the difference between the proportion of data where an illusion does not exist and the proportion of data where an illusion exists as the first reward information, or the server can determine the proportion of data where an illusion exists as the first reward information. In addition, there are various other methods for determining the first reward information, which may vary depending on the actual application scenario. This specification does not specifically limit these methods in the embodiments.

[0065] The server can determine the second reward information based on the matching degree between the second detection result and the transaction risk label.

[0066] Based on the first reward information and / or the second reward information, a target reward information is determined, and the second detection model is trained using reinforcement learning based on the target reward information. The server can determine the target reward information as the mean (or maximum, minimum, weighted sum, etc.) between the first reward information and the second reward information.

[0067] Thus, as Figure 5 As shown, in the reinforcement learning phase of the second detection model, the trained target model (i.e., the hallucination detection model) can serve as a key component of the multi-dimensional scoring system. It is used to evaluate whether the reasoning results generated by the main policy model (i.e., the second detection model) (i.e., the fourth hallucination detection result) are strictly based on the input data, thus preventing the model from learning and reinforcing erroneous patterns of fabricated facts. This mechanism ensures that the model's reinforcement learning process not only focuses on the correctness of the conclusions but also emphasizes the data fidelity of the reasoning process.

[0068] Specifically, the main strategy model can generate multiple inference paths in parallel for the same input. The illusion detection model can perform illusion detection on the inference results (i.e., multiple second detection results) corresponding to the multiple inference paths to obtain a fourth illusion detection result. The server can determine the first reward information based on the proportion of data without illusions, the proportion of data with illusions, and the proportion of data with different illusion types in the fourth illusion detection result. Based on the second detection results and the transaction risk label, the server can determine the second reward information. Finally, the server can update the parameters of the second detection model based on the target reward information (i.e., reward signal) determined by the first reward information and / or the second reward information. In this way, the model parameters can be optimized along the "data → rule → conclusion" path.

[0069] In practical applications, the specific processing method for determining the target illusion label corresponding to the transaction risk detection result based on the illusion detection label, the first illusion detection result, and the second illusion detection result in step S208 above can vary. The following provides one optional processing method, such as... Figure 6 As shown, the specific process may include the following steps S2082 to S20810.

[0070] In step S2082, it is determined whether the hallucination detection label matches the first hallucination detection result.

[0071] In step S2084, if the hallucination detection label matches the first hallucination detection result, the hallucination detection label is determined as the target hallucination label corresponding to the transaction risk detection result.

[0072] In step S2086, if the hallucination detection label does not match the first hallucination detection result, it is determined whether the hallucination detection label matches the second hallucination detection result.

[0073] In step S2088, if the hallucination detection label matches the second hallucination detection result, the hallucination detection label is determined as the target hallucination label corresponding to the transaction risk detection result.

[0074] In step S20810, if the hallucination detection label does not match the second hallucination detection result, the first training data, the transaction risk detection result, the hallucination detection label, the first hallucination detection result, and the second hallucination detection result are sent to the preset reviewer, and the target hallucination label corresponding to the transaction risk detection result is determined according to the return result of the preset reviewer.

[0075] In implementation, such as Figure 7 As shown, the server can first determine whether the forward inference result of the large language model (i.e., the first illusion detection result) matches the illusion detection label (i.e., perform label verification on the forward inference result of the large language model). If they match, the illusion detection label can be directly determined as the target illusion label corresponding to the transaction risk detection result. If they do not match, the server can further determine whether the reverse inference result of the large language model (i.e., the second illusion detection result) matches the illusion detection label. If they match, the illusion detection label can be directly determined as the target illusion label corresponding to the transaction risk detection result. If they do not match, the target illusion label can be determined by manual annotation.

[0076] In addition, the server can also determine the illusion detection type corresponding to the transaction detection result through the above matching process. The illusion detection type can include simple type, complex type and high-quality type.

[0077] For example, such as Figure 7 As shown, if the hallucination detection label matches the first hallucination detection result, the hallucination detection type corresponding to the transaction detection result can be a simple type. If the hallucination detection label matches the second hallucination detection result, the hallucination detection type corresponding to the transaction detection result can be a complex type. If the target hallucination label corresponding to the transaction detection result is determined by manual annotation, then the hallucination detection type corresponding to the transaction detection result can be a high-quality type.

[0078] Then, the server can build a training sample pool based on the transaction detection results corresponding to different hallucination detection types. In this way, the server can select target training data that meets the training requirements of the target model from the training sample pool according to the training requirements of the target model and the hallucination detection type corresponding to the transaction detection results, so as to conduct targeted training of the target model based on the target training data.

[0079] For example, if the training requirement of the target model is to achieve balance, the server can select target training data from the training sample pool based on the number of transaction detection results corresponding to each illusion detection type. Specifically, assuming that the number of transaction detection results corresponding to the simple type, complex type, and high-quality type are 100, 200, and 300 respectively, the service can select target training data from the transaction detection results corresponding to these three illusion detection types based on a first selection number, wherein the first selection number can be no greater than 100.

[0080] Alternatively, if the training requirement for the target model is high quality, the server can select target training data from the transaction detection results corresponding to the high quality type based on a second quantity, wherein the second quantity may not be greater than the number of transaction detection results corresponding to the high quality requirement type.

[0081] In practical applications, the transaction risk detection result can be obtained by preprocessing the first risk detection result corresponding to the first training data. The preprocessing can include format conversion processing and / or feature restoration processing. The format conversion processing can be used to convert the format of the first risk detection result into the data processing format of a preset large language model. The feature restoration processing can be used to replace the target feature in the first risk detection result with the corresponding feature in the first training data. The target feature can be the feature in the first risk detection result whose matching degree with the first training data is lower than a preset matching degree threshold.

[0082] In implementation, such as Figure 7 As shown, the server can perform pre-processing, that is, after obtaining the first risk detection result, the server can perform pre-processing on the first risk detection result to obtain the first training data.

[0083] For the first risk detection result containing free text input with diverse formats, in order to better detect hallucination problems, the first risk detection result can be processed as follows: Figure 8 The standardized preprocessing shown is as follows: Assuming the data processing format of the large language model is standard JSON, the server can first determine whether the format of the first risk detection result is standard JSON (i.e., whether it is formatted input). If not, data structuring processing can be performed, that is, the format of the first risk detection result can be converted into standard JSON format. If yes, it can further determine whether the key point references in the first risk detection result are original features (i.e., whether the first risk detection result contains features that do not match the first training data). If yes, generalized feature restoration processing can be performed on the first risk detection result, that is, the target feature in the first risk detection result is replaced with the corresponding feature in the first training data.

[0084] In this way, format conversion and feature restoration processing can facilitate subsequent illusion detection. For example, after feature restoration processing, the knowledge anchor verification method can be used to map the transaction detection results in the first training data to a structured knowledge graph and perform structured verification on the transaction detection results, which can help locate possible illusion problems in the transaction detection results (such as incorrect feature references or descriptions that do not match the facts).

[0085] By using data preprocessing methods such as knowledge anchors and data anchors, data standardization can be achieved, enabling dynamic adaptation to different input data in various business scenarios and ensuring the generalizability of the hallucination detection model.

[0086] In practical applications, the preset prompts may include character settings, domain knowledge of the first training data, and a hallucination detection process. The hallucination detection process may include detection steps and detection rules for different types of hallucinations.

[0087] In implementation, due to the different scenarios to which historical business data belongs, different constraint methods for prompt information can be adopted for different business scenarios and different hallucination detection requirements. The reasoning ability of a large language model can be used for hallucination detection. Specifically, during the forward reasoning process (i.e., determining the first hallucination detection result) and the backward reasoning process, different prompt information can be designed according to different reasoning types. For example... Figure 9 As shown, a prompt message can consist of the following elements: (1) Role setting: identity, task, and evaluation content; (2) Domain knowledge of the first training data (i.e., injection of specific domain knowledge): domain knowledge and precautions; (3) Hallucination detection process: detection steps, as well as the detection focus and detection rules that include the detection steps; (4) Input data: data structure and data format; (5) Output format requirements: output elements and output format.

[0088] In practical applications, the trained target model can be used to detect whether the detection results output by the risk detection model contain illusions. Based on the detection results, transaction risk detection processing can be performed on the trading object. The specific processing methods for transaction risk detection can be varied; the following provides one optional processing method, such as... Figure 10 As shown, the specific process may include the following steps S1002 to S1010.

[0089] In step S1002, a transaction risk detection request for the target transaction object is received.

[0090] The target object can be any user, account, or other object that can engage in transaction activities.

[0091] In practice, the server can identify users who generate transaction behavior within a preset detection period as target objects. Alternatively, the server can identify a user as a target object and trigger a transaction risk detection request for that target object when it detects that the user has triggered a transaction such as resource transfer.

[0092] In step S1004, in response to the transaction risk detection request, the transaction business data of the target transaction object is obtained.

[0093] In implementation, in response to a transaction risk detection request, the server can obtain the transaction business data of the target transaction object. The transaction business data may include transaction data (such as transaction time, transaction platform, transaction channel, transaction object, quantity of transaction resources, etc.), graph structure data corresponding to the target transaction object (used to represent the historical transaction relationship between the target transaction object and other transaction objects), and equipment information of the target transaction object used to trigger the target transaction business (such as equipment identifier, equipment model, frequency of use, etc.).

[0094] In step S1006, a pre-trained risk detection model is used to perform transaction risk detection processing on the transaction business data to obtain a third detection result.

[0095] The risk detection model can be a model built based on a preset deep learning algorithm.

[0096] In step S1008, the trained target model is used to perform hallucination detection processing on the third detection result to obtain the target detection result.

[0097] In step S1010, based on the target detection result, it is determined whether the third detection result is a hallucination, and if it is determined that the third detection result is not a hallucination, it is determined whether the target trading object has a trading risk based on the third detection result.

[0098] In practice, the server can determine whether there is a transaction risk in the target transaction object based on the third detection result if the third detection result does not contain any hallucinations. Alternatively, the server can determine whether there is a transaction risk in the target transaction object based on the data in the third detection result that does not contain hallucinations and based on the hallucination type corresponding to the third detection result if the third detection result does contain hallucinations.

[0099] Thus, as Figure 11As shown, the server can construct a detection system based on a data processing layer, a model illusion detection layer, and a detection result feedback and application layer. In the data processing layer, the server can input historical business data into a pre-trained discriminant model to obtain the first risk detection result corresponding to the historical business data. This first risk detection result is then used as the detection content for preprocessing, such as determining whether the first risk detection result is a structured input and whether it contains non-original features, to perform format conversion and feature restoration processing on the first risk detection result. Furthermore, the first risk detection result output by the discriminant model can include a detection conclusion regarding the existence of transaction risk, and the corresponding COT (Content Token) chain for that detection conclusion.

[0100] In the model illusion detection layer, the server can use preset prompts to introduce domain knowledge of the first training data to identify logical illusions through knowledge anchors. That is, in the process of using the preset large language model for illusion detection, in addition to historical business data and corresponding transaction risk results, domain knowledge can also be input into the large language model as key knowledge to detect whether there are illusion problems such as factual illusion, faithful illusion and logical illusion in the transaction risk detection results.

[0101] In the detection result feedback and application layer, after determining the target illusion label corresponding to the transaction risk detection result, training can be performed based on the transaction risk detection result and the corresponding target model, such as... Figure 11 As shown, the trained target model can be used for high-quality training sample generation and processing, reinforcement learning reward training, and production chain quality inspection and other application scenarios.

[0102] Hallucination detection models are not only a crucial part of quality control in the production chain, but can also be applied to the reinforcement learning stage of other strategy models (such as discriminative inference models and message generation models). Hallucination detection models can serve as a key component of multi-dimensional scoring systems, used to evaluate whether the inference results generated by the main strategy model (such as discriminative inference models and message generation models) are strictly based on the input data, preventing the model from learning and reinforcing erroneous patterns of fabricated facts. This mechanism ensures that the model's reinforcement learning process not only focuses on the correctness of the conclusions but also emphasizes the data fidelity of the inference process.

[0103] In the high-quality training sample generation and processing scenario, the trained target model can be used to filter out sample data that does not contain hallucinations, and use it as training data for the transaction risk detection model (such as the second detection model).

[0104] In reinforcement learning reward training scenarios, the trained target model can perform hallucination detection on the model output to determine reward information based on the hallucination detection results, and then perform reinforcement learning training on the model based on the reward information.

[0105] In the production chain quality inspection scenario, the trained target model can be used to perform hallucination detection on the model output results. Based on the hallucination detection results, it can be determined whether the model output results are qualified. If qualified, the downstream tasks can be processed based on the model output results (such as transaction risk assessment). If unqualified, the model output results can be sent as negative samples to the high-quality training sample generation and processing scenario.

[0106] In this way, by establishing a standardized hallucination detection system and classifying hallucinations into three types (such as factual hallucinations, faithful hallucinations, and logical hallucinations), the detection system can be composed of a three-layer link, capable of dynamically adapting to different input data and various hallucination problems under various business scenarios and automatically iterating. Through the construction of a three-layer system, there is no need for customized processing of scenarios, which can improve the generalization of the model and the system.

[0107] In addition, a three-layer system link is used, which includes data preprocessing, hallucination classification, knowledge injection, and sample difficulty classification. A deep learning model is used for hallucination detection. The target model can learn complex hallucination problems from a larger language model and continuously update and optimize. Compared with traditional rule recognition and manual judgment methods, the accuracy and recall of multi-class hallucination detection using the trained target model are maintained at a high level on various scene evaluation sets, which improves the adaptability and scalability of the detection system.

[0108] Preprocessing ensures the model can adapt to data input from various transaction scenarios. By filtering sample data using the first analysis model and the trained target model, larger-scale models can be used to generate training data and perform knowledge transfer. This allows knowledge from a large first analysis model to be transferred to a smaller second analysis model, reducing computational resource consumption, maintaining high inference speed and low computational cost, and improving system scalability and real-time performance. Simultaneously, while ensuring model performance, training efficiency is improved, and a single illusion detection model can adapt to all scenarios, reducing deployment costs.

[0109] This specification provides a data processing method that can acquire first training data, which may include historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results. Using a preset large language model, a first illusion detection result is determined based on the historical business data and the transaction risk detection results. Using the preset large language model, a second illusion detection result is determined based on preset prompt information, historical business data, the transaction risk detection results, and the corresponding illusion detection labels. The preset prompt information can be used to constrain the preset large language model to detect whether the transaction risk detection results are illusory given the illusion detection labels. Based on the illusion detection labels, the first illusion detection result, and the second illusion detection result, a target illusion label corresponding to the transaction risk detection result is determined. Based on the transaction risk detection results and the corresponding target illusion label, a target model is trained to detect whether the transaction risk detection results output by the preset model are illusory. In this way, by using a pre-set large language model, and through model forward inference (i.e., performing hallucination detection without giving hallucination detection labels) and model backward inference (i.e., performing hallucination detection with given hallucination detection labels), the presence of hallucinations in the transaction risk detection results can be detected. This ensures the accuracy of the target hallucination label determined based on the hallucination detection label, the first hallucination detection result, and the second hallucination detection result. Furthermore, by using the transaction risk detection results and the corresponding target hallucination label, the hallucination detection effect of the target model can be improved, thereby improving the accuracy of transaction risk detection based on the trained target model.

[0110] The above describes the data processing method provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide a data processing device, such as... Figure 12 As shown.

[0111] The data processing device includes: a first acquisition module 1201, a first detection module 1202, a second detection module 1203, a label determination module 1204, and a model training module 1205, wherein: The first acquisition module 1201 is used to acquire first training data, the first training data including historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results; The first detection module 1202 is used to determine the first illusion detection result based on the historical business data and the transaction risk detection result using a preset large language model; The second detection module 1203 is used to determine the second illusion detection result by using the preset large language model based on the preset prompt information, the historical business data, the transaction risk detection result and the corresponding illusion detection label. The preset prompt information is used to constrain the preset large language model to detect whether the transaction risk detection result is an illusion given the illusion detection label. The label determination module 1204 is used to determine the target illusion label corresponding to the transaction risk detection result based on the illusion detection label, the first illusion detection result, and the second illusion detection result; The model training module 1205 is used to train the target model based on the transaction risk detection results and the corresponding target illusion labels, so as to detect whether there is an illusion in the transaction risk detection results output by the preset model based on the trained target model.

[0112] In the embodiments described in this specification, the device further includes: The second acquisition module is used to acquire the second training data; The third detection module is used to perform transaction risk detection on the second training data using the first detection model to obtain the first detection result; The fourth detection module is used to use the trained target model to detect whether the first detection result is a hallucination, and obtain a third hallucination detection result. The data filtering module is used to filter the second training data based on the third hallucination detection result to obtain the third training data; The model training module is used to train the second detection model based on the third training data, so as to detect whether there is a transaction risk in the transaction object to be detected based on the trained second detection model, wherein the number of parameters of the second detection model is smaller than the number of parameters of the first detection model.

[0113] In the embodiments of this specification, the model training module is used for: Obtain the transaction risk label corresponding to the third training data; Using the second detection model, transaction risk detection is performed on the third training data to obtain a second detection result; Using the trained target model, the presence of hallucination in the second detection result is detected to obtain the fourth hallucination detection result; The second detection model is trained based on the fourth illusion detection result, the second detection result, and the transaction risk label.

[0114] In the embodiments of this specification, the model training module is used for: Based on the fourth hallucination detection result, determine the proportion of data in the second detection result that do not contain hallucinations and / or contain hallucinations, and determine the first reward information; Based on the matching degree between the second detection result and the transaction risk label, the second reward information is determined; Based on the first reward information and / or the second reward information, target reward information is determined, and based on the target reward information, the second detection model is trained using reinforcement learning.

[0115] In this embodiment of the specification, the label determination module 1204 is used for: Determine whether the hallucination detection label matches the first hallucination detection result; If the hallucination detection label matches the first hallucination detection result, then the hallucination detection label is determined as the target hallucination label corresponding to the transaction risk detection result; If the hallucination detection label does not match the first hallucination detection result, then determine whether the hallucination detection label matches the second hallucination detection result; If the hallucination detection label matches the second hallucination detection result, then the hallucination detection label is determined as the target hallucination label corresponding to the transaction risk detection result; If the hallucination detection label does not match the second hallucination detection result, the first training data, the transaction risk detection result, the hallucination detection label, the first hallucination detection result, and the second hallucination detection result are sent to a preset reviewer, and the target hallucination label corresponding to the transaction risk detection result is determined based on the return result of the preset reviewer.

[0116] In this embodiment of the specification, the transaction risk detection result is obtained by preprocessing the first risk detection result corresponding to the first training data. The preprocessing includes format conversion processing and / or feature restoration processing. The format conversion processing is used to convert the format of the first risk detection result into the data processing format of the preset large language model. The feature restoration processing is used to replace the target feature in the first risk detection result with the corresponding feature in the first training data. The target feature is the feature in the first risk detection result whose matching degree with the first training data is lower than a preset matching degree threshold.

[0117] In the embodiments of this specification, the preset prompt information includes character settings, domain knowledge of the first training data, and a hallucination detection process. The hallucination detection process includes detection steps and detection rules for different types of hallucinations.

[0118] In the embodiments described in this specification, the device further includes: The request receiving module is used to receive transaction risk detection requests for the target transaction object; The data acquisition module is used to acquire the transaction business data of the target transaction object in response to the transaction risk detection request; The fifth detection module is used to perform transaction risk detection processing on the transaction business data using a pre-trained risk detection model to obtain the third detection result; The sixth detection module is used to perform hallucination detection processing on the third detection result using the trained target model to obtain the target detection result; The risk detection module is used to determine whether the third detection result is a hallucination based on the target detection result, and if it is determined that the third detection result is not a hallucination, it determines whether the target trading object has a trading risk based on the third detection result.

[0119] This specification provides a data processing apparatus that can acquire first training data, which may include historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results. Using a preset large language model, based on the historical business data and the transaction risk detection results, a first illusion detection result is determined. Using the preset large language model, based on preset prompt information, historical business data, transaction risk detection results, and corresponding illusion detection labels, a second illusion detection result is determined. The preset prompt information can be used to constrain the preset large language model to detect whether the transaction risk detection result is an illusion given an illusion detection label. Based on the illusion detection label, the first illusion detection result, and the second illusion detection result, a target illusion label corresponding to the transaction risk detection result is determined. Based on the transaction risk detection result and the corresponding target illusion label, a target model is trained to detect whether the transaction risk detection result output by the preset model is an illusion. In this way, by using a pre-set large language model, and through model forward inference (i.e., performing hallucination detection without giving hallucination detection labels) and model backward inference (i.e., performing hallucination detection with given hallucination detection labels), the presence of hallucinations in the transaction risk detection results can be detected. This ensures the accuracy of the target hallucination label determined based on the hallucination detection label, the first hallucination detection result, and the second hallucination detection result. Furthermore, by using the transaction risk detection results and the corresponding target hallucination label, the hallucination detection effect of the target model can be improved, thereby improving the accuracy of transaction risk detection based on the trained target model.

[0120] The above are the data processing apparatuses provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide a data processing device, such as... Figure 13 As shown.

[0121] The data processing device can provide terminal equipment or servers, etc., for the above embodiments.

[0122] like Figure 13 As shown, device 1300 mainly consists of a communication interface 1302, a user interface 1304, a processor 1306, and a data storage 1308. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 1313. The communication interface 1302 enables device 1300 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 1302 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 1302 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 1302 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 1302 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces.

[0123] User interface 1304 includes receiving user input and providing output to the user. Therefore, user interface 1304 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 1304 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 1304 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 1300 may support remote access from other devices via communication interface 1302 or another physical interface (not shown). User interface 1304 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 1304 can also be configured as a display device for rendering or displaying text fragments.

[0124] Processor 1306 may include one or more general-purpose processors and / or special-purpose processors.

[0125] Data storage 1308 may include one or more volatile storage components and may be integrated wholly or partially with processor 1306. Data storage 1308 may include removable and non-removable components.

[0126] Processor 1306 is capable of executing program instructions 1318 (e.g., compiled or uncompiled program logic and / or machine code) stored in data store 1308 to perform the various functions described herein. Data store 1308 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 1300, enable device 1300 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 1318 by processor 1306 may result in processor 1306 using data 1312.

[0127] For example, program instructions 1318 may include an operating system 1322 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 1300 and one or more applications 1320 (e.g., a browser, social application, or game application). Similarly, data 1312 may include operating system data 1316 and application data 1313. Operating system data 1316 is primarily accessible to the operating system 1322, while application data 1313 is primarily accessible to one or more applications 1320. Application data 1313 may reside in a file system visible or hidden from the user of device 1300.

[0128] Application 1320 can communicate with operating system 1312 through one or more application programming interfaces (APIs). These APIs help application 1320 read and / or write application data 1313, transmit or receive information via communication interface 1302, receive or display information on user interface 1304, etc.

[0129] In some terminology, application 1320 may be simply referred to as "app". Furthermore, application 1320 can be downloaded to device 1300 through one or more online app stores or app markets. However, applications can also be installed on device 1300 in other ways, such as through a web browser or a physical interface on device 1300 (e.g., a USB port).

[0130] Specifically, in this embodiment, the data processing device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the data processing device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following: Obtain first training data, which includes historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results; Using a pre-defined large language model, based on the historical business data and the transaction risk detection results, the first illusion detection result is determined; Using the preset large language model, based on the preset prompt information, the historical business data, the transaction risk detection result, and the corresponding hallucination detection label, a second hallucination detection result is determined. The preset prompt information is used to constrain the preset large language model to detect whether the transaction risk detection result is a hallucination given the hallucination detection label. Based on the illusion detection label, the first illusion detection result, and the second illusion detection result, the target illusion label corresponding to the transaction risk detection result is determined; Based on the transaction risk detection results and the corresponding target illusion labels, the target model is trained to detect whether the transaction risk detection results output by the preset model are illusory.

[0131] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the data processing device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0132] This specification provides a data processing device that can acquire first training data, which may include historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results. Using a preset large language model, based on the historical business data and the transaction risk detection results, a first illusion detection result is determined. Using the preset large language model, based on preset prompt information, historical business data, transaction risk detection results, and corresponding illusion detection labels, a second illusion detection result is determined. The preset prompt information can be used to constrain the preset large language model to detect whether the transaction risk detection result is an illusion given an illusion detection label. Based on the illusion detection label, the first illusion detection result, and the second illusion detection result, a target illusion label corresponding to the transaction risk detection result is determined. Based on the transaction risk detection result and the corresponding target illusion label, a target model is trained to detect whether the transaction risk detection result output by the preset model is an illusion. In this way, by using a pre-set large language model, and through model forward inference (i.e., performing hallucination detection without giving hallucination detection labels) and model backward inference (i.e., performing hallucination detection with given hallucination detection labels), the presence of hallucinations in the transaction risk detection results can be detected. This ensures the accuracy of the target hallucination label determined based on the hallucination detection label, the first hallucination detection result, and the second hallucination detection result. Furthermore, by using the transaction risk detection results and the corresponding target hallucination label, the hallucination detection effect of the target model can be improved, thereby improving the accuracy of transaction risk detection based on the trained target model.

[0133] Furthermore, based on the above Figures 1 to 11 This specification also provides a storage medium for storing computer-executable instruction information in one or more embodiments. In one specific embodiment, the storage medium may be a USB flash drive, optical disc, hard disk, etc. When the computer-executable instruction information stored in the storage medium is executed by a processor, it can realize the following process: Obtain first training data, which includes historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results; Using a pre-defined large language model, based on the historical business data and the transaction risk detection results, the first illusion detection result is determined; Using the preset large language model, based on the preset prompt information, the historical business data, the transaction risk detection result, and the corresponding hallucination detection label, a second hallucination detection result is determined. The preset prompt information is used to constrain the preset large language model to detect whether the transaction risk detection result is a hallucination given the hallucination detection label. Based on the illusion detection label, the first illusion detection result, and the second illusion detection result, the target illusion label corresponding to the transaction risk detection result is determined; Based on the transaction risk detection results and the corresponding target illusion labels, the target model is trained to detect whether the transaction risk detection results output by the preset model are illusory.

[0134] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described storage medium embodiment is basically similar to the method embodiment, so the description is relatively simple; relevant parts can be referred to the description of the method embodiment.

[0135] This specification provides a storage medium that can acquire first training data, which may include historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results. Using a preset large language model, a first illusion detection result is determined based on the historical business data and the transaction risk detection results. Using the preset large language model, a second illusion detection result is determined based on preset prompt information, historical business data, the transaction risk detection results, and the corresponding illusion detection labels. The preset prompt information can be used to constrain the preset large language model to detect whether the transaction risk detection results are illusory given the illusion detection labels. Based on the illusion detection labels, the first illusion detection result, and the second illusion detection result, a target illusion label corresponding to the transaction risk detection result is determined. Based on the transaction risk detection results and the corresponding target illusion label, a target model is trained to detect whether the transaction risk detection results output by the preset model are illusory. In this way, by using a pre-set large language model, and through model forward inference (i.e., performing hallucination detection without giving hallucination detection labels) and model backward inference (i.e., performing hallucination detection with given hallucination detection labels), the presence of hallucinations in the transaction risk detection results can be detected. This ensures the accuracy of the target hallucination label determined based on the hallucination detection label, the first hallucination detection result, and the second hallucination detection result. Furthermore, by using the transaction risk detection results and the corresponding target hallucination label, the hallucination detection effect of the target model can be improved, thereby improving the accuracy of transaction risk detection based on the trained target model.

[0136] Furthermore, based on the above Figures 1 to 11 This specification also provides one or more embodiments of a computer program product, including a computer program, which, when executed by a processor, can perform the following processes: Obtain first training data, which includes historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results; Using a pre-defined large language model, based on the historical business data and the transaction risk detection results, the first illusion detection result is determined; Using the preset large language model, based on the preset prompt information, the historical business data, the transaction risk detection result, and the corresponding hallucination detection label, a second hallucination detection result is determined. The preset prompt information is used to constrain the preset large language model to detect whether the transaction risk detection result is a hallucination given the hallucination detection label. Based on the illusion detection label, the first illusion detection result, and the second illusion detection result, the target illusion label corresponding to the transaction risk detection result is determined; Based on the transaction risk detection results and the corresponding target illusion labels, the target model is trained to detect whether the transaction risk detection results output by the preset model are illusory.

[0137] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described embodiment of a computer program product is relatively simple in description because it is fundamentally similar to the method embodiment; relevant parts can be referred to the description of the method embodiment.

[0138] This specification provides a computer program product that can acquire first training data, which may include historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results. Using a preset large language model, based on the historical business data and the transaction risk detection results, a first illusion detection result is determined. Using the preset large language model, based on preset prompt information, historical business data, transaction risk detection results, and corresponding illusion detection labels, a second illusion detection result is determined. The preset prompt information can be used to constrain the preset large language model to detect whether the transaction risk detection result is an illusion given an illusion detection label. Based on the illusion detection label, the first illusion detection result, and the second illusion detection result, a target illusion label corresponding to the transaction risk detection result is determined. Based on the transaction risk detection result and the corresponding target illusion label, a target model is trained to detect whether the transaction risk detection result output by the preset model is an illusion. In this way, by using a pre-set large language model, and through model forward inference (i.e., performing hallucination detection without giving hallucination detection labels) and model backward inference (i.e., performing hallucination detection with given hallucination detection labels), the presence of hallucinations in the transaction risk detection results can be detected. This ensures the accuracy of the target hallucination label determined based on the hallucination detection label, the first hallucination detection result, and the second hallucination detection result. Furthermore, by using the transaction risk detection results and the corresponding target hallucination label, the hallucination detection effect of the target model can be improved, thereby improving the accuracy of transaction risk detection based on the trained target model.

[0139] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0140] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0141] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0142] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0143] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0144] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The embodiments described herein are illustrated with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 Figure 1 The steps of the function specified in one or more boxes.

[0148] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0149] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0150] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0151] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitations, the presence of additional identical elements in the process, method, article, or apparatus that includes said elements is not excluded.

[0152] It should also be noted that the terms "one," "an," and "the" do not specifically refer to the singular; they can also include the plural. Ordinal numbers such as "first," "second," etc., do not necessarily indicate order; often they are used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0153] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving data sent by B, or it can be understood as A indirectly receiving data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending data directly to A, or it can be understood as B indirectly sending data to A through other entities such as C. Here, C can be a single subject, or two or more subjects.

[0154] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations within this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0155] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

[0156] It should be noted that the user data obtained in this manual is authorized by the user and does not involve user privacy.

[0157] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0158] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0159] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0160] The above description is merely an embodiment of this specification and is not intended to limit this document. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A data processing method, comprising: Obtain first training data, which includes historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results; Using a pre-defined large language model, based on the historical business data and the transaction risk detection results, the first illusion detection result is determined; Using the preset large language model, based on the preset prompt information, the historical business data, the transaction risk detection result, and the corresponding hallucination detection label, a second hallucination detection result is determined. The preset prompt information is used to constrain the preset large language model to detect whether the transaction risk detection result is a hallucination given the hallucination detection label. Based on the illusion detection label, the first illusion detection result, and the second illusion detection result, the target illusion label corresponding to the transaction risk detection result is determined; Based on the transaction risk detection results and the corresponding target illusion labels, the target model is trained to detect whether the transaction risk detection results output by the preset model are illusory.

2. The method according to claim 1, further comprising: Obtain the second training data; Using the first detection model, transaction risk detection is performed on the second training data to obtain the first detection result; Using the trained target model, the presence of hallucination in the first detection result is detected to obtain the third hallucination detection result; Based on the third hallucination detection result, the second training data is filtered to obtain the third training data; The second detection model is trained based on the third training data, and the trained second detection model is used to detect whether there is a transaction risk in the transaction object to be detected. The number of parameters in the second detection model is smaller than the number of parameters in the first detection model.

3. The method according to claim 2, wherein training the second detection model based on the third training data comprises: Obtain the transaction risk label corresponding to the third training data; Using the second detection model, transaction risk detection is performed on the third training data to obtain a second detection result; Using the trained target model, the presence of hallucination in the second detection result is detected to obtain the fourth hallucination detection result; The second detection model is trained based on the fourth illusion detection result, the second detection result, and the transaction risk label.

4. The method according to claim 3, wherein training the second detection model based on the fourth illusion detection result, the second detection result, and the transaction risk label comprises: Based on the fourth hallucination detection result, determine the proportion of data in the second detection result that do not contain hallucinations and / or contain hallucinations, and determine the first reward information; Based on the matching degree between the second detection result and the transaction risk label, the second reward information is determined; Based on the first reward information and / or the second reward information, target reward information is determined, and based on the target reward information, the second detection model is trained using reinforcement learning.

5. The method according to claim 1, wherein determining the target illusion label corresponding to the transaction risk detection result based on the illusion detection label, the first illusion detection result, and the second illusion detection result includes: Determine whether the hallucination detection label matches the first hallucination detection result; If the hallucination detection label matches the first hallucination detection result, then the hallucination detection label is determined as the target hallucination label corresponding to the transaction risk detection result; If the hallucination detection label does not match the first hallucination detection result, then determine whether the hallucination detection label matches the second hallucination detection result; If the hallucination detection label matches the second hallucination detection result, then the hallucination detection label is determined as the target hallucination label corresponding to the transaction risk detection result; If the hallucination detection label does not match the second hallucination detection result, the first training data, the transaction risk detection result, the hallucination detection label, the first hallucination detection result, and the second hallucination detection result are sent to a preset reviewer, and the target hallucination label corresponding to the transaction risk detection result is determined based on the return result of the preset reviewer.

6. The method according to claim 1, wherein the transaction risk detection result is obtained by preprocessing the first risk detection result corresponding to the first training data, the preprocessing includes format conversion processing and / or feature restoration processing, the format conversion processing is used to convert the format of the first risk detection result into the data processing format of the preset large language model, and the feature restoration processing is used to replace the target feature in the first risk detection result with the corresponding feature in the first training data, wherein the target feature is the feature in the first risk detection result whose matching degree with the first training data is lower than a preset matching degree threshold.

7. The method according to claim 1, wherein the preset prompt information includes character settings, domain knowledge of the first training data, and a hallucination detection process, wherein the hallucination detection process includes detection steps and detection rules for different types of hallucinations.

8. The method according to claim 1, further comprising: Receive transaction risk detection requests for the target trading entity; In response to the transaction risk detection request, obtain the transaction business data of the target transaction object; Using a pre-trained risk detection model, the transaction data is processed to detect transaction risks, resulting in a third detection result. Using the trained target model, hallucination detection processing is performed on the third detection result to obtain the target detection result; Based on the target detection results, it is determined whether the third detection result is a hallucination, and if it is determined that the third detection result is not a hallucination, it is determined whether the target trading object has a trading risk based on the third detection result.

9. A data processing apparatus, comprising: The first acquisition module is used to acquire the first training data, which includes historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results. The first detection module is used to determine the first illusion detection result based on the historical business data and the transaction risk detection result using a preset large language model; The second detection module is used to determine the second illusion detection result by using the preset large language model based on the preset prompt information, the historical business data, the transaction risk detection result and the corresponding illusion detection label. The preset prompt information is used to constrain the preset large language model to detect whether the transaction risk detection result is an illusion given the illusion detection label. The label determination module is used to determine the target illusion label corresponding to the transaction risk detection result based on the illusion detection label, the first illusion detection result, and the second illusion detection result; The model training module is used to train the target model based on the transaction risk detection results and the corresponding target illusion labels, so as to detect whether the transaction risk detection results output by the preset model are illusory based on the trained target model.

10. A data processing apparatus, the data processing apparatus comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to: Obtain first training data, which includes historical business data, transaction risk detection results corresponding to the historical business data, and illusion detection labels corresponding to the transaction risk detection results; Using a pre-defined large language model, based on the historical business data and the transaction risk detection results, the first illusion detection result is determined; Using the preset large language model, based on the preset prompt information, the historical business data, the transaction risk detection result, and the corresponding hallucination detection label, a second hallucination detection result is determined. The preset prompt information is used to constrain the preset large language model to detect whether the transaction risk detection result is a hallucination given the hallucination detection label. Based on the illusion detection label, the first illusion detection result, and the second illusion detection result, the target illusion label corresponding to the transaction risk detection result is determined; Based on the transaction risk detection results and the corresponding target illusion labels, the target model is trained to detect whether the transaction risk detection results output by the preset model are illusory.

Images