Trade data verification method and device, equipment and medium
By extracting and processing text and image data in trade data, using deep learning models to fusion features and inputting large language models, the problems of low efficiency and inaccurate verification of trade data in the existing technology are solved, and more efficient and accurate data verification is achieved.
Patent Information
- Application Number
- CN202411694843.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-03-14
AI Technical Summary
In the prior art, trade data verification is inefficient and inaccurate enough, and manual intervention is required to cause time-consuming and labor-intensive and easy to introduce human errors.
By extracting text data and picture data from the target trade data, using the residual network for graph encoding, combining small Transformer models to fuse text and picture features, building feature extraction requests and inputting large language models to achieve more accurate data verification.
It improves the efficiency and accuracy of trade data verification, reduces the occurrence of human errors, and achieves a more efficient and reliable data verification process.
Smart Images
Figure CN119624366A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a trade data verification method, device, equipment and medium. Background Art
[0002] In order to maintain the stability of the trading market and reduce transaction risks, it is usually necessary to verify the data in the trading process to safeguard the interests of both parties to the transaction.
[0003] In the existing technology, manual intervention and verification are usually required, and manpower is used to review trade data section by section, which is not only time-consuming and labor-intensive, but also prone to human errors, resulting in inaccurate verification results. Summary of the invention
[0004] The embodiments of the present application provide a trade data verification method, device, equipment and medium, which aim to solve the problem of low efficiency and inaccuracy of trade data verification.
[0005] In a first aspect, an embodiment of the present application provides a trade data verification method, which includes:
[0006] In response to a verification instruction for target trade data, extracting target text data and target image data from the target trade data;
[0007] Convert the target text data into an embedding vector to obtain a first vector;
[0008] Inputting the target image data into a residual network for graphic encoding to obtain a graphic block embedding vector;
[0009] Inputting the graph block embedding vector into the first fully connected layer to obtain a graph global embedding vector;
[0010] Inputting the first vector and the global embedding vector of the graph into a small Transformer model to obtain a second vector;
[0011] Inputting the graphic block embedding vector into the second fully connected layer to obtain a third vector;
[0012] constructing an element extraction request according to the first vector, the second vector, and the third vector, and inputting the element extraction request into a large language model to obtain a target element of the target trade data;
[0013] The target trade data is verified according to the target elements to obtain a verification result.
[0014] In a second aspect, an embodiment of the present application provides a trade data verification device, which includes:
[0015] An extraction unit, configured to extract target text data and target image data from the target trade data in response to a verification instruction for the target trade data;
[0016] A conversion unit, used for converting the target text data into an embedding vector to obtain a first vector;
[0017] An input unit, used for inputting the target image data into a residual network for graphic encoding to obtain a graphic block embedding vector;
[0018] The input unit is further used to input the graphic block embedding vector into the first fully connected layer to obtain a graphic global embedding vector;
[0019] The input unit is further used to input the first vector and the global embedding vector of the graph into a small Transformer model to obtain a second vector;
[0020] The input unit is further used to input the graphic block embedding vector into the second fully connected layer to obtain a third vector;
[0021] The input unit is further used to construct an element extraction request according to the first vector, the second vector, and the third vector, and input the element extraction request into the large language model to obtain the target element of the target trade data;
[0022] The verification unit is used to verify the target trade data according to the target elements to obtain a verification result.
[0023] In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the trade data verification method described in the first aspect when executing the computer program.
[0024] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the trade data verification method described in the first aspect above.
[0025] The embodiments of the present application provide a trade data verification method, device, equipment and medium. The present application inputs target image data in the target trade data into a residual network for graphic encoding to extract all detailed features of the image; inputs the graphic block embedding vector into the first fully connected layer, and can extract a graphic global embedding vector with smaller dimension and more general information; inputs the first vector and the graphic global embedding vector obtained by converting the target text data into a small Transformer model to obtain a second vector, which can fully integrate text features and image features; constructs an element extraction request based on the third vector obtained after the first vector, the second vector and the graphic block embedding vector are processed by the second fully connected layer, which can further realize the full integration of text features and image features, so that the large language model can extract elements more accurately, thereby assisting in more accurate and efficient trade data verification. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0027] Figure 1 A schematic diagram of an application scenario of the trade data verification method provided in an embodiment of the present application;
[0028] Figure 2 A flow chart of a trade data verification method provided in an embodiment of the present application;
[0029] Figure 3 A schematic block diagram of a trade data verification device provided in an embodiment of the present application;
[0030] Figure 4 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0032] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0033] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0034] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0035] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of the trade data verification method provided in an embodiment of the present application; Figure 2 A flow chart of a trade data verification method provided in an embodiment of the present application. The trade data verification method is applied in a server and is executed by application software installed in the server.
[0036] like Figure 2 As shown, the method includes steps S101 to S108.
[0037] S101. In response to a verification instruction for target trade data, extract target text data and target image data from the target trade data.
[0038] In this embodiment, the technical solution is described with the server as the execution subject. The user terminal (such as a smart phone, tablet computer and other intelligent terminals) used by the user can interact with the server for data. Specifically, the server provides a trade data verification platform, and the user can log in to the trade data verification platform using the user terminal. The user interaction interface of the trade data verification platform is displayed on the terminal interface of the user terminal, and there is at least one data upload interface in the user interaction interface. When the user selects the data upload interface and uploads the trade data to the server, the subsequent trade data verification can be performed in the server.
[0039] In this embodiment, the verification instruction may be automatically triggered when data is detected to be uploaded through the interface.
[0040] In this embodiment, the target trade data may be various documents, invoices and other data generated during transactions between enterprises.
[0041] In this embodiment, the target text data and the target image data in the target trade data may be extracted by using the OCR (Optical Character Recognition) technology.
[0042] S102: Convert the target text data into an embedding vector to obtain a first vector.
[0043] In this embodiment, the target text data may be converted into an embedding vector by using methods such as One-Hot Encoding, TF-IDF (Term Frequency-Inverse Document Frequency), Word2Vec, GloVe (Global Vectors for Word Representation), and FastText.
[0044] Specifically, the conversion algorithm may be selected according to the actual usage scenario.
[0045] For example, for training of large-scale corpora, the GloVe method can be used; for morphologically rich data, the FastText method can be used to process sub-word features.
[0046] S103, inputting the target image data into a residual network (Residual Network, ResNet) for graphic encoding to obtain a graphic block embedding vector.
[0047] This embodiment inputs the target image data into the residual network for graphic encoding, which can convert the image into image features and obtain a numerical matrix containing all the detailed features of the image, namely the graphic block embedding vector, to facilitate model recognition.
[0048] Of course, in other embodiments, a ViT (Vision Transformer) model may be used to replace the residual network for graphics encoding.
[0049] S104: Input the graphic block embedding vector into the first fully connected layer to obtain a graphic global embedding vector.
[0050] In this embodiment, the graphic block embedding vector is input into the first fully connected layer, so as to obtain a graphic global embedding vector with more general information and smaller dimension.
[0051] S105: Input the first vector and the global embedding vector of the graphic into a small Transformer model to obtain a second vector.
[0052] In this embodiment, the small Transformer model may include a 3-layer transformer or a 4-layer transformer.
[0053] It should be noted that the traditional Transformer model includes 12 layers of transformers. However, in this embodiment, it has been verified that the use of 3-layer transformers or 4-layer transformers can achieve the best feature processing effect while making the model more lightweight.
[0054] S106: Input the graphic block embedding vector into the second fully connected layer to obtain a third vector.
[0055] In this embodiment, by inputting the graphic block embedding vector into the second fully connected layer, it is possible to reduce feature complexity while retaining effective features, thereby achieving an information compression effect, thereby facilitating subsequent processing.
[0056] S107: construct an element extraction request according to the first vector, the second vector, and the third vector, and input the element extraction request into a large language model to obtain target elements of the target trade data.
[0057] In this embodiment, constructing a feature extraction request according to the first vector, the second vector, and the third vector includes:
[0058] The first vector, the second vector, and the third vector are added to the initial prompt word embedding vector to obtain the verification request.
[0059] Through the above embodiments, it is possible to fully integrate text features and image features, so that the large language model can extract elements more accurately, thereby assisting in more accurate and efficient trade data verification.
[0060] In this embodiment, before inputting the element extraction request into the large language model, the method further includes:
[0061] Acquire data consisting of text and pictures to construct a first training sample;
[0062] Marking the first training sample by elements, and extracting elements of the first training sample by using the large language model;
[0063] Comparing the extracted elements with the marked elements to obtain a comparison result;
[0064] The large language model is fine-tuned according to the comparison result until the extracted elements are identical to the marked elements.
[0065] For example, the initial instructions can be "describe the content of the picture", "describe the objects that appear in the picture", "output text OCR results", etc., that is, given the standard output of the large language model, the large language model is allowed to use this as a label to continuously optimize the model parameters so that the large language model is familiar with the meaning contained in the picture embedding vector; the instructions are further refined, and the large language model is aligned with the picture embedding vector through fine-tuning of the instructions so that the large language model has a more accurate understanding of the picture. For example, the instructions can be further adjusted to questions such as "What is the quantity of product X?" and "What is the total amount?" to further mine the picture information until the extracted elements are the same as the marked elements.
[0066] Furthermore, the method further comprises:
[0067] When the extracted elements are the same as the marked elements, acquiring historical trade data to construct a second training sample; wherein the second training sample includes trade element marks;
[0068] Fixing the parameters of the large language model and the parameters of the residual network, and using the second training sample to train the first fully connected layer, the second fully connected layer and the small Transformer model;
[0069] When the elements extracted by the large language model are the same as the elements corresponding to the trade element tags, the training is stopped to obtain the initial prompt word embedding vector.
[0070] In the above embodiment, after the large language model can recognize valid information in the picture, the entire model system is further optimized using data in the corresponding field, the parameters of the large language model and the residual network are fixed, and the parameters of the first fully connected layer, the second fully connected layer, and the small Transformer model are adjusted to learn the optimal prompt word embedding vector prompt embedding, as well as the optimal parameters of the first fully connected layer, the second fully connected layer, and the small Transformer model.
[0071] In the above embodiment, a small Transformer structure is used to fuse text embedding (embedding vector) and image embedding (i.e., the first vector and the global embedding vector of the graphic are input into the small Transformer model to obtain the second vector), and the projection of the text embedding and the image embedding is input into the large language model (i.e., a feature extraction request is constructed according to the first vector, the second vector, and the third vector, and the feature extraction request is input into the large language model). This can capture the global information of the image and obtain the details of the image, allowing the model to better understand the image. At the same time, during the training process, the parameters of the large language model and the image encoder (i.e., the residual network) are fixed, and only the transformer component and the fully connected layer are trained, which effectively reduces the training cost.
[0072] In the above embodiment, through continuous optimization training, the large language model can more accurately extract the elements of trade data, overcoming the problem that the existing large language model cannot well understand data with both pictures and text (for example, the current common large language model cannot effectively process data with pictures such as road signs and signboards and more text).
[0073] In this embodiment, after extracting the target text data and the target image data from the target trade data, the method further includes:
[0074] When the information amount of the target image data is less than or equal to a preset threshold, constructing the feature extraction request according to the first vector, the second vector, and the graphic block embedding vector; or
[0075] The feature extraction request is constructed according to the first vector and the third vector.
[0076] The preset threshold value may be configured according to model performance and the like.
[0077] In the above embodiment, for images that are not information-dense, the feature extraction request can be directly constructed based on the first vector, the second vector, and the graphic block embedding vector, or the feature extraction request can be constructed based on the first vector and the third vector. Although this weakens the model's ability to understand image and text information, it can make the entire model system more lightweight and can meet the feature processing requirements of images with less information.
[0078] S108. Verify the target trade data according to the target elements to obtain a verification result.
[0079] In this embodiment, the target trade data is verified according to the target element to obtain a verification result, including:
[0080] Obtaining a pre-constructed element list of the target trade data;
[0081] traversing the element list using the target element;
[0082] When the target element does not match the element list, determining that the verification result is that the target trade data has a risk; or
[0083] When the target element matches the element list, it is determined that the verification result is that there is no risk in the target trade data.
[0084] The element list may be constructed based on reliable information such as transaction contracts to serve as a verification benchmark.
[0085] Among them, when the target element does not match the element list, it means that the elements do not correspond one to one, and at this time, there may be transaction risks.
[0086] Furthermore, after obtaining the verification result, the verification result can be sent to the terminal device of a designated person to assist in further verification and confirmation, thereby improving the verification efficiency of trade data and avoiding errors introduced due to human factors, thereby improving the security of trade.
[0087] It can be seen from the above technical solutions that the present application inputs the target image data in the target trade data into the residual network for graphic encoding to extract all the detailed features of the image; the graphic block embedding vector is input into the first fully connected layer, so as to extract the graphic global embedding vector with smaller dimension and more general information; the first vector converted from the target text data and the graphic global embedding vector are input into the small Transformer model to obtain the second vector, which can fully integrate the text features and the image features; the feature extraction request is constructed according to the third vector obtained after the first vector, the second vector and the graphic block embedding vector are processed by the second fully connected layer, which can further realize the full integration of the text features and the image features, so that the large language model can extract features more accurately, thereby assisting in more accurate and efficient trade data verification.
[0088] The present application also provides a trade data verification device, which is used to execute any embodiment of the aforementioned trade data verification method. Figure 3 , Figure 3 It is a schematic block diagram of the trade data verification device 100 provided in an embodiment of the present application.
[0089] Among them, Figure 3As shown, the trade data verification device 100 includes an extraction unit 101 , a conversion unit 102 , an input unit 103 , and a verification unit 104 .
[0090] The extraction unit 101 is used to extract target text data and target image data from the target trade data in response to a verification instruction for the target trade data.
[0091] In this embodiment, the technical solution is described with the server as the execution subject. The user terminal (such as a smart phone, tablet computer and other intelligent terminals) used by the user can interact with the server for data. Specifically, the server provides a trade data verification platform, and the user can log in to the trade data verification platform using the user terminal. The user interaction interface of the trade data verification platform is displayed on the terminal interface of the user terminal, and there is at least one data upload interface in the user interaction interface. When the user selects the data upload interface and uploads the trade data to the server, the subsequent trade data verification can be performed in the server.
[0092] In this embodiment, the verification instruction may be automatically triggered when data is detected to be uploaded through the interface.
[0093] In this embodiment, the target trade data may be various documents, invoices and other data generated during transactions between enterprises.
[0094] In this embodiment, the target text data and the target image data in the target trade data may be extracted by using the OCR (Optical Character Recognition) technology.
[0095] The conversion unit 102 is used to convert the target text data into an embedded vector to obtain a first vector.
[0096] In this embodiment, the target text data may be converted into an embedding vector by using methods such as One-Hot Encoding, TF-IDF (Term Frequency-Inverse Document Frequency), Word2Vec, GloVe (Global Vectors for Word Representation), and FastText.
[0097] Specifically, the conversion algorithm may be selected according to the actual usage scenario.
[0098] For example, for training of large-scale corpora, the GloVe method can be used; for morphologically rich data, the FastText method can be used to process sub-word features.
[0099] The input unit 103 is used to input the target image data into a residual network (Residual Network, ResNet) for graphic encoding to obtain a graphic block embedding vector.
[0100] This embodiment inputs the target image data into the residual network for graphic encoding, which can convert the image into image features and obtain a numerical matrix containing all the detailed features of the image, namely the graphic block embedding vector, to facilitate model recognition.
[0101] Of course, in other embodiments, a ViT (Vision Transformer) model may be used to replace the residual network for graphics encoding.
[0102] The input unit 103 is further used to input the graphic block embedding vector into the first fully connected layer to obtain a graphic global embedding vector.
[0103] In this embodiment, the graphic block embedding vector is input into the first fully connected layer, so as to obtain a graphic global embedding vector with more general information and smaller dimension.
[0104] The input unit 103 is further used to input the first vector and the graphic global embedding vector into the small Transformer model to obtain a second vector.
[0105] In this embodiment, the small Transformer model may include a 3-layer transformer or a 4-layer transformer.
[0106] It should be noted that the traditional Transformer model includes 12 layers of transformers. However, in this embodiment, it has been verified that the use of 3-layer transformers or 4-layer transformers can achieve the best feature processing effect while making the model more lightweight.
[0107] The input unit 103 is further used to input the graphic block embedding vector into the second fully connected layer to obtain a third vector.
[0108] In this embodiment, by inputting the graphic block embedding vector into the second fully connected layer, it is possible to reduce feature complexity while retaining effective features, thereby achieving an information compression effect, thereby facilitating subsequent processing.
[0109] The input unit 103 is further used to construct an element extraction request according to the first vector, the second vector, and the third vector, and input the element extraction request into the large language model to obtain the target element of the target trade data.
[0110] In this embodiment, constructing a feature extraction request according to the first vector, the second vector, and the third vector includes:
[0111] The first vector, the second vector, and the third vector are added to the initial prompt word embedding vector to obtain the verification request.
[0112] Through the above embodiments, it is possible to fully integrate text features and image features, so that the large language model can extract elements more accurately, thereby assisting in more accurate and efficient trade data verification.
[0113] In this embodiment, before inputting the element extraction request into the large language model, data consisting of text and pictures are obtained to construct a first training sample;
[0114] Marking the first training sample by elements, and extracting elements of the first training sample by using the large language model;
[0115] Comparing the extracted elements with the marked elements to obtain a comparison result;
[0116] The large language model is fine-tuned according to the comparison result until the extracted elements are identical to the marked elements.
[0117] For example, the initial instructions can be "describe the content of the picture", "describe the objects that appear in the picture", "output text OCR results", etc., that is, given the standard output of the large language model, the large language model is allowed to use this as a label to continuously optimize the model parameters so that the large language model is familiar with the meaning contained in the picture embedding vector; the instructions are further refined, and the large language model is aligned with the picture embedding vector through fine-tuning of the instructions so that the large language model has a more accurate understanding of the picture. For example, the instructions can be further adjusted to questions such as "What is the quantity of product X?" and "What is the total amount?" to further mine the picture information until the extracted elements are the same as the marked elements.
[0118] Further, when the extracted elements are the same as the marked elements, historical trade data is obtained to construct a second training sample; wherein the second training sample includes trade element marks;
[0119] Fixing the parameters of the large language model and the parameters of the residual network, and using the second training sample to train the first fully connected layer, the second fully connected layer and the small Transformer model;
[0120] When the elements extracted by the large language model are the same as the elements corresponding to the trade element tags, the training is stopped to obtain the initial prompt word embedding vector.
[0121] In the above embodiment, after the large language model can recognize valid information in the picture, the entire model system is further optimized using data in the corresponding field, the parameters of the large language model and the residual network are fixed, and the parameters of the first fully connected layer, the second fully connected layer, and the small Transformer model are adjusted to learn the optimal prompt word embedding vector prompt embedding, as well as the optimal parameters of the first fully connected layer, the second fully connected layer, and the small Transformer model.
[0122] In the above embodiment, a small Transformer structure is used to fuse text embedding (embedding vector) and image embedding (i.e., the first vector and the global embedding vector of the graphic are input into the small Transformer model to obtain the second vector), and the projection of the text embedding and the image embedding is input into the large language model (i.e., a feature extraction request is constructed according to the first vector, the second vector, and the third vector, and the feature extraction request is input into the large language model). This can capture the global information of the image and obtain the details of the image, allowing the model to better understand the image. At the same time, during the training process, the parameters of the large language model and the image encoder (i.e., the residual network) are fixed, and only the transformer component and the fully connected layer are trained, which effectively reduces the training cost.
[0123] In the above embodiment, through continuous optimization training, the large language model can more accurately extract the elements of trade data, overcoming the problem that the existing large language model cannot well understand data with both pictures and text (for example, the current common large language model cannot effectively process data with pictures such as road signs and signboards and more text).
[0124] In this embodiment, after extracting the target text data and the target image data from the target trade data, when the amount of information of the target image data is less than or equal to a preset threshold, constructing the element extraction request according to the first vector, the second vector, and the graphic block embedding vector; or
[0125] The feature extraction request is constructed according to the first vector and the third vector.
[0126] The preset threshold value may be configured according to model performance and the like.
[0127] In the above embodiment, for images that are not information-dense, the feature extraction request can be directly constructed based on the first vector, the second vector, and the graphic block embedding vector, or the feature extraction request can be constructed based on the first vector and the third vector. Although this weakens the model's ability to understand image and text information, it can make the entire model system more lightweight and can meet the feature processing requirements of images with less information.
[0128] The verification unit 104 is used to verify the target trade data according to the target element to obtain a verification result.
[0129] In this embodiment, the verification unit 104 verifies the target trade data according to the target element to obtain a verification result, including:
[0130] Obtaining a pre-constructed element list of the target trade data;
[0131] traversing the element list using the target element;
[0132] When the target element does not match the element list, determining that the verification result is that the target trade data has a risk; or
[0133] When the target element matches the element list, it is determined that the verification result is that there is no risk in the target trade data.
[0134] The element list may be constructed based on reliable information such as transaction contracts to serve as a verification benchmark.
[0135] Among them, when the target element does not match the element list, it means that the elements do not correspond one to one, and at this time, there may be transaction risks.
[0136] Furthermore, after obtaining the verification result, the verification result can be sent to the terminal device of a designated person to assist in further verification and confirmation, thereby improving the verification efficiency of trade data and avoiding errors introduced due to human factors, thereby improving the security of trade.
[0137] It can be seen from the above technical solutions that the present application inputs the target image data in the target trade data into the residual network for graphic encoding to extract all the detailed features of the image; the graphic block embedding vector is input into the first fully connected layer, so as to extract the graphic global embedding vector with smaller dimension and more general information; the first vector converted from the target text data and the graphic global embedding vector are input into the small Transformer model to obtain the second vector, which can fully integrate the text features and the image features; the feature extraction request is constructed according to the third vector obtained after the first vector, the second vector and the graphic block embedding vector are processed by the second fully connected layer, which can further realize the full integration of the text features and the image features, so that the large language model can extract features more accurately, thereby assisting in more accurate and efficient trade data verification.
[0138] The above trade data verification device can be implemented in the form of a computer program. The computer program can be implemented in Figure 4 Runs on the computer device shown.
[0139] See also Figure 4 , Figure 4 4 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 400 is a server, or a server cluster. The server can be an independent server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0140] See also Figure 4 The computer device 400 includes a processor 402 , a memory, and a network interface 405 connected via a device bus 401 , wherein the memory may include a storage medium 403 and an internal memory 404 .
[0141] The storage medium 403 may store an operating system 4031 and a computer program 4032. When the computer program 4032 is executed, the processor 402 may execute a trade data verification method.
[0142] The processor 402 is used to provide computing and control capabilities to support the operation of the entire computer device 400 .
[0143] The internal memory 404 provides an environment for the operation of the computer program 4032 in the storage medium 403. When the computer program 4032 is executed by the processor 402, the processor 402 can execute the trade data verification method.
[0144] The network interface 405 is used for network communication, such as providing data information transmission, etc. Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 400 to which the solution of the present application is applied. The specific computer device 400 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0145] The processor 402 is used to run a computer program 4032 stored in the memory to implement the trade data verification method disclosed in the embodiment of the present application.
[0146] Those skilled in the art will understand that Figure 4 The embodiments of the computer device shown in the figure do not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the computer device may only include a memory and a processor. In such embodiments, the structure and function of the memory and the processor are the same as those of the embodiment of the present invention. Figure 4 The embodiments shown are consistent and will not be described again here.
[0147] It should be understood that in the embodiment of the present application, the processor 402 may be a central processing unit (CPU), and the processor 402 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0148] In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the trade data verification method disclosed in the embodiment of the present application is implemented.
[0149] It should be noted that the data involved in this case were all obtained legally.
[0150] It will be clear to those skilled in the art that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It will be appreciated by those of ordinary skill in the art that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.
[0151] In the several embodiments provided in the present application, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, and units with the same function may be combined into one unit. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.
[0152] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.
[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a background server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk.
[0155] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A trade data verification method, characterized in that: include: In response to a verification instruction for target trade data, extracting target text data and target image data from the target trade data; Convert the target text data into an embedding vector to obtain a first vector; Inputting the target image data into a residual network for graphic encoding to obtain a graphic block embedding vector; Inputting the graph block embedding vector into the first fully connected layer to obtain a graph global embedding vector; Inputting the first vector and the global embedding vector of the graph into a small Transformer model to obtain a second vector; Inputting the graphic block embedding vector into the second fully connected layer to obtain a third vector; constructing an element extraction request according to the first vector, the second vector, and the third vector, and inputting the element extraction request into a large language model to obtain a target element of the target trade data; The target trade data is verified according to the target elements to obtain a verification result.
2. The trade data verification method according to claim 1, characterized in that: The small Transformer model includes 3 transformer layers or 4 transformer layers.
3. The trade data verification method according to claim 1, characterized in that: The constructing a feature extraction request according to the first vector, the second vector, and the third vector includes: The first vector, the second vector, and the third vector are added to the initial prompt word embedding vector to obtain the verification request.
4. The trade data verification method according to claim 3, characterized in that: Before inputting the element extraction request into the large language model, the method further includes: Acquire data consisting of text and pictures to construct a first training sample; Marking the first training sample by elements, and extracting elements of the first training sample by using the large language model; Comparing the extracted elements with the marked elements to obtain a comparison result; The large language model is fine-tuned according to the comparison result until the extracted elements are identical to the marked elements.
5. The trade data verification method according to claim 4, characterized in that: The method further comprises: When the extracted elements are the same as the marked elements, acquiring historical trade data to construct a second training sample; wherein the second training sample includes trade element marks; Fixing the parameters of the large language model and the parameters of the residual network, and using the second training sample to train the first fully connected layer, the second fully connected layer and the small Transformer model; When the elements extracted by the large language model are the same as the elements corresponding to the trade element tags, the training is stopped to obtain the initial prompt word embedding vector.
6. The trade data verification method according to claim 1, characterized in that: The verifying the target trade data according to the target element to obtain a verification result includes: Obtaining a pre-constructed element list of the target trade data; traversing the element list using the target element; When the target element does not match the element list, determining that the verification result is that the target trade data has a risk; or When the target element matches the element list, it is determined that the verification result is that there is no risk in the target trade data.
7. The trade data verification method according to claim 1, characterized in that: After extracting the target text data and the target image data from the target trade data, the method further includes: When the information amount of the target image data is less than or equal to a preset threshold, constructing the feature extraction request according to the first vector, the second vector, and the graphic block embedding vector; or The feature extraction request is constructed according to the first vector and the third vector.
8. A trade data verification device, characterized in that: include: An extraction unit, configured to extract target text data and target image data from the target trade data in response to a verification instruction for the target trade data; A conversion unit, used for converting the target text data into an embedding vector to obtain a first vector; An input unit, used for inputting the target image data into a residual network for graphic encoding to obtain a graphic block embedding vector; The input unit is further used to input the graphic block embedding vector into the first fully connected layer to obtain a graphic global embedding vector; The input unit is further used to input the first vector and the global embedding vector of the graph into a small Transformer model to obtain a second vector; The input unit is further used to input the graphic block embedding vector into the second fully connected layer to obtain a third vector; The input unit is further used to construct an element extraction request according to the first vector, the second vector, and the third vector, and input the element extraction request into the large language model to obtain the target element of the target trade data; The verification unit is used to verify the target trade data according to the target elements to obtain a verification result.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the trade data verification method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the trade data verification method according to any one of claims 1 to 7.