Trade background auditing method, device and equipment and storage medium

Abstract

The invention discloses a trade background auditing method, device and equipment and a storage medium, and is applied to the field of financial science and technology, and the method comprises the steps: obtaining multi-modal trade background data of a customer, the multi-modal trade background data comprises text trade background data, image trade background data and video trade background data; extracting different parameters from the trade background data according to modals through a trade background information extraction model; and scoring the extracted parameters, and determining a trade background auditing result for the customer according to a score value. Multi-modal trade background data combining texts, images and videos are audited through a trade background information extraction model, associated information between pictures and characters in provided materials is extracted, data missing is avoided, personnel are liberated from redundant files, only important information is concerned, and the efficiency is improved. And the working efficiency and the auditing precision of the handling personnel can be improved.

Description

Technical Field

[0001] The present invention relates to the field of fintech, and particularly to a method, device, equipment and storage medium for trade background review. Background Art

[0002] When a customer handles online pledge invoicing of a financial asset pool, the personnel of the handling bank need to review the customer's trade background and submit it to the authorized person for review. Currently, the review of the customer's trade background is all carried out by the personnel of the handling bank manually reviewing the trade background data in text form.

[0003] However, since the trade background data includes a large amount of data such as copies of commercial drafts, commodity trading contracts, and value-added tax invoices, manual review by the personnel of the handling bank often requires a large amount of manpower and time. In addition, there is also a risk of subjective will. And the review is limited to the text form, which will inevitably result in the lack of information, thus affecting the accuracy of the trade background review. Summary of the Invention

[0004] The present invention provides a method for trade background review to achieve the review of the customer's trade background.

[0005] According to a first aspect of the present invention, there is provided a method for trade background review, including: obtaining multi-modal trade background data of a customer, wherein the multi-modal trade background data includes text trade background data, image trade background data, and video trade background data;

[0006] Extracting different parameters from the trade background data by a trade background information extraction model according to the modality;

[0007] Scoring the extracted parameters and determining the trade background review result for the customer according to the score value.

[0008] Optionally, obtaining multi-modal trade background data of a customer includes:

[0009] Receiving a trade background review request and opening a multi-modal data receiving port according to the trade background review request, wherein the multi-modal data receiving port includes a text port, an image port, and a video port;

[0010] Receiving the text trade background data through the text port, wherein the text trade background data includes a transaction contract, a copy of a draft, and a value-added tax invoice;

[0011] Receiving the image trade background data through the image port, wherein the image trade background data includes a business license, product pictures, and site pictures;

[0012] Receiving the video trade background data through the video port, wherein the video trade background data includes a product production process video and a product usage instruction video.

[0013] Optionally, before extracting different parameters from the trade background data by the trade background information extraction model according to the modality, it further includes:

[0014] Performing key frame extraction on the video trade background data to obtain image trade background data;

[0015] Performing image preprocessing operations on the image trade background data to obtain preprocessed image trade background data, wherein the image preprocessing operations include image cropping and image enhancement;

[0016] Performing text preprocessing operations on the text trade background data to obtain preprocessed text trade background data, wherein the text preprocessing operations include special character removal and word segmentation operations.

[0017] Optionally, extracting different parameters from the trade background data by the trade background information extraction model according to the modality includes:

[0018] Converting the preprocessed text trade background data into a text vector by the trade background information extraction model, and converting the preprocessed image trade background data into an image vector;

[0019] Performing self-attention weighted calculation on the text vector to obtain an updated text vector;

[0020] Performing feature extraction on the image vector to obtain an updated image vector, wherein the dimensions of the updated image vector and the updated text vector are the same;

[0021] Performing parameter extraction according to the updated text vector and the updated image vector, wherein the parameters include entities, objects, and link relationships.

[0022] Optionally, performing parameter extraction according to the updated text vector and the updated image vector includes:

[0023] Fusing the updated text vector and the updated image vector to obtain a fused vector;

[0024] Extracting entities from the updated text vector, extracting objects from the updated image vector, and extracting link relationships from the fused vector.

[0025] According to another aspect of the present invention, there is provided a trade background auditing device, including: a trade background data acquisition module for acquiring the trade background data of a customer in multiple modalities, wherein the modalities include text, image, and video;

[0026] A parameter extraction module, configured to extract different parameters from the trade background data according to modalities through a trade background information extraction model;

[0027] An audit module, configured to score the extracted parameters and determine a trade background audit result for the customer according to the score value.

[0028] According to another aspect of the present invention, there is provided an electronic device, including:

[0029] At least one processor; and

[0030] A memory communicatively connected to the at least one processor; wherein,

[0031] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method according to any embodiment of the present invention.

[0032] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the method according to any embodiment of the present invention when executed.

[0033] According to another aspect of the present invention, there is provided a computer program product including a computer program which, when executed by a processor, implements the method according to any embodiment of the present invention.

[0034] The beneficial technical effect of the present invention is that through the trade background information extraction model, the trade background data combining text, images and videos is audited, the association information between the pictures and the text in the provided materials is extracted, data loss is avoided, and personnel are liberated from redundant files and only focus on important information, which helps to improve the work efficiency and audit accuracy of the personnel in the handling bank.

[0035] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0037] Figure 1 It is a flowchart of a trade background review method provided according to Embodiment 1 of the present invention;

[0038] Figure 2 It is a flowchart of a trade background review method provided according to Embodiment 2 of the present invention;

[0039] Figure 3 It is a schematic structural diagram of a trade background review device provided according to Embodiment 3 of the present invention;

[0040] Figure 4 It is a schematic structural diagram of an electronic device provided to implement Embodiment 4 of the present invention. Detailed implementation manners

[0041] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices. In addition, the collected information is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data, etc., all comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse

[0043] Embodiment 1

[0044] Figure 1FIG. 0 is a flowchart of a trade background review method provided by Embodiment 1 of the present invention. This embodiment is applicable to the situation of reviewing the trade background of customers. This method can be executed by a trade background review device, which can be implemented in the form of hardware and / or software. As Figure 1 shown, the method includes:

[0045] Step S101, obtaining multi-modal trade background data of the customer.

[0046] Optionally, obtaining multi-modal trade background data of the customer includes: receiving a trade background review request and opening a multi-modal data receiving port according to the trade background review request, where the multi-modal data receiving port includes a text port, an image port, and a video port; receiving text trade background data through the text port, where the text trade background data includes a transaction contract, a copy of a bill of exchange, and a value-added tax invoice; receiving image trade background data through the image port, where the image trade background data includes a business license, product pictures, and site pictures; receiving video trade background data through the video port, where the video trade background data includes a product production process video and a product usage instruction video.

[0047] Specifically, the multi-modal trade background data in this embodiment includes text trade background data, image trade background data, and video trade background data. There is a human-computer interaction interface on the terminal device. When financial institution business handlers need to review the trade background of customers, they can send a trade background review request on the human-computer interaction interface. When the trade background review request is received, a multi-modal data receiving port will be opened, and a data upload interface will be presented on the interaction interface. Different positions on the upload interface correspond to receiving ports for different modal data, such as a text port, an image port, and a video port. Different modal trade background data uploaded will be received through different ports, such as receiving text trade background data through the text port, receiving image trade background data through the image port, and receiving video trade background data through the video port. Of course, only three ports are used as examples in this embodiment. In actual applications, different numbers of receiving ports can be set according to different needs. For example, the text port can be further divided into a contract port and an invoice port, etc. Of course, only examples are given in this embodiment, and the number of receiving ports set on the data upload interface is not limited. As long as different modes of trade background data can be accurately received, it is within the protection scope of this application.

[0048] Optionally, before extracting different parameters from the trade background data by the trade background information extraction model according to the modality, it further includes: extracting key frames from the video trade background data to obtain image trade background data; performing image preprocessing operations on the image trade background data to obtain preprocessed image trade background data, where the image preprocessing operations include image cropping and image enhancement; performing text preprocessing operations on the text trade background data to obtain preprocessed text trade background data, where the text preprocessing operations include special character removal and word segmentation operations.

[0049] Among them, in this embodiment, after collecting trade background data of different modalities, since the collected data has multiple sources and the picture sizes or text formats are different, it is difficult to directly apply them to the trade background information extraction model. Therefore, it is necessary to preprocess the obtained multi-modal trade background data, and the preprocessing methods adopted for trade background data of different modalities are different. For example, for text trade background data, such as transaction contracts, bill copies, or VAT invoices, etc., it is necessary to extract the text, then remove the special characters in the extracted text, and perform word segmentation operations on the removed text to obtain preprocessed text trade background data; for image trade background data, such as business licenses, product pictures, and venue pictures, the pictures will be classified first, then the pictures of different sizes will be adjusted to a unified size, and the pictures will be cropped and noise removed. Finally, the pictures with noise removed will be image-enhanced to enhance the clarity of the pictures; for video trade background data, it is necessary to extract the key frames in the video and downsample the extracted key frames. The key frame pictures extracted can be used as new image trade background data, and the image trade background data obtained by key frame extraction can be preprocessed in the same way as the directly collected image trade background data, which will not be elaborated in this embodiment.

[0050] Step S102, extract different parameters from the trade background data by the trade background information extraction model according to the modality.

[0051] It should be noted that in this embodiment, before the trade background information extraction model extracts parameters from the trade background data according to the modality, the trade background information extraction model will first be trained using samples, and the obtained training samples are also multimodal, including sample text trade background data, sample image trade background data, sample video trade background data, etc. Moreover, preprocessing operations will be performed on the sample text trade background data, sample image trade background data, and sample video trade background data. The preprocessing operation methods are roughly the same as those for the above-mentioned multimodal trade background data to be reviewed, which will not be elaborated in this embodiment. After preprocessing, sample text vectors and sample image vectors will be obtained. For the sample text vectors, entity recognition and relationship annotation will be performed. For the sample image vectors, object detection and annotation will be carried out, and the entities and objects will be associated to establish the association relationship between text and image. The above-mentioned annotated trade background data will be used as a sample data set to train the trade background information extraction model.

[0052] Optionally, different parameters are extracted from the trade background data by the trade background information extraction model according to the modality, including: converting the preprocessed text trade background data into text vectors by the trade background information extraction model, and converting the preprocessed image trade background data into image vectors; performing self-attention weighted calculation on the text vectors to obtain updated text vectors; performing feature extraction on the image vectors to obtain updated image vectors, where the dimensions of the updated image vectors and the updated text vectors are the same; extracting parameters according to the updated text vectors and the updated image vectors, where the parameters include entities, objects, and link relationships.

[0053] Specifically, in this embodiment, when preprocessing the multi-modal trade background data, the preprocessed trade background data will be transmitted to the trade background information extraction model. The trade background information extraction model in this embodiment mainly includes an input module, a modality fusion module, an information extraction module, and an output module. In this embodiment, specifically, the preprocessed trade background data is transmitted to the input module of the model, and the input module converts the received data into a format that can be processed by the model. For example, the preprocessed text trade background data is converted into a text vector, and the preprocessed image trade background data is converted into an image vector. Specifically, the text vector is a two-dimensional vector, and the image vector is a three-dimensional vector. That is, the representation of trade background data in different modalities is converted into a vector format that can be processed by the model, and the text vector and the image vector are input into the modality fusion module. Of course, in this embodiment, only one text vector and one image vector are obtained as an example for illustration. In actual applications, due to the extremely large amount of multi-modal trade background data, a large number of text vectors and image vectors can be obtained. In this embodiment, the specific number of text vectors and image vectors obtained is not limited.

[0054] Among them, the modality fusion module in this embodiment is used to fuse the representation vectors of trade background data in different modalities to establish a global multi-modal representation, and the text vector and the image vector will be processed first before fusion. For the text vector, the self-attention weighted algorithm is used for calculation to obtain the correlation between each element in the text vector and all other elements, so as to capture richer context information, and a weighted vector is generated accordingly. The generated weighted vector is used as the updated text vector. The self-attention weighted algorithm involves related operations such as query, similarity calculation, weight assignment, and weighted summation. Since the principle of the self-attention weighted algorithm is not the focus of this application, it will not be elaborated in this embodiment. For the image vector, the dimension remains unchanged at two-dimensional. For the image vector, a convolutional neural network can be used for feature extraction, and an updated image vector is generated based on the extracted features. The updated image vector generated based on the extracted features is a two-dimensional vector. Since the dimensions of the updated text vector and the updated image vector are the same, both being two-dimensional vectors, the updated text vector and the updated image vector can be fused to obtain a fusion vector, and the fusion vector contains the link relationship between the image and the text.

[0055] Optionally, parameter extraction is performed according to the updated text vector and the updated image vector, including: fusing the updated text vector and the updated image vector to obtain a fusion vector; extracting entities from the updated text vector, extracting objects from the updated image vector, and extracting link relationships from the fusion vector.

[0056] Specifically, in this embodiment, after the representation vectors of different modality data are fused by the modality fusion module, the updated text vector, the updated image vector, and the fusion vector can all be input into the information extraction module. Among them, the information extraction module of this embodiment can extract target information from the received vectors. For example, entities can be extracted from the updated text vector, such as company names, objects can be extracted from the updated image vector, such as company addresses, and link relationships can be extracted from the fusion vector. Of course, this is only an example in this embodiment, and the types of parameters extracted from each vector are not limited.

[0057] Step S103: Score the extracted parameters, and determine the trade background review result for the customer according to the score value.

[0058] Optionally, scoring the extracted parameters includes: obtaining the weights corresponding to the entity, the object, and the link relationship; performing weighted calculation on the entity, the object, and the link relationship based on the weights to obtain the score value.

[0059] Optionally, determining the trade background review result for the customer according to the score value includes: judging whether the score value exceeds a preset threshold. If so, it is determined that the trade background data is true and the customer trade background review passes; otherwise, it is determined that the trade background data is false and the customer trade background review fails.

[0060] Specifically, in this embodiment, after extracting specified parameters, such as entities, objects, and link relationships, from the trade background data, each specified parameter will be normalized to obtain specified parameters with a unified unit. At the same time, the weights corresponding to each specified parameter will also be obtained. For example, the weight corresponding to the entity is a, the weight corresponding to the object is b, and the weight corresponding to the link relationship is c. The specific numerical values of the weights corresponding to each specified parameter are not limited in this embodiment, and the score value is obtained by using the following formula (1):

[0061] Score value = a * entity + b * object + c * link relationship (1)

[0062] Among them, in this embodiment, after obtaining the score value by scoring based on the extracted parameters, the obtained score value will also be compared with the configured preset threshold. For example, when the obtained score value is 9 points and the preset threshold set in advance is 10 points, since the score value is less than the preset threshold, it can be determined that the customer trade background data submitted by the current customer is false, so the customer trade background review fails; when the obtained score value is 11, since the score value is greater than the preset threshold, it can be determined that the customer trade background data submitted by the current customer is true, so the customer trade background review passes.

[0063] It should be noted that when the customer's trade background review is passed, the subsequent business application process of the customer will continue; when the customer's trade background review fails, it indicates that the trade background data submitted by the customer is false. At this time, a prompt message "The trade background review of this customer fails" will be generated, and the business application process that the customer is currently handling will be aborted, thus ensuring the security of the assets of financial institutions. For example, when reviewing the trade background data submitted by a customer for a financing application, when it is determined through the review that the customer's trade background review fails, the loan to the customer will be aborted, and the customer will also be marked. When the customer applies for other business in a financial institution later, it will be reviewed according to more stringent review rules, thus further ensuring the security of the assets of financial institutions.

[0064] It is worth mentioning that information extraction is a text processing technology and an important task in natural language processing. It aims to extract specific types of entities, relationships, events, and other factual information from natural language. With the rapid development of deep learning, its excellent feature selection and extraction have had an important impact on information extraction, shifting from traditional manual feature extraction to deep learning-based methods, using word vectors in the text as input, and completing the information extraction task through end-to-end feature extraction by neural networks. Multimodal information extraction belongs to multimodal learning and information extraction technology. Traditional information extraction mainly focuses on extracting entities and relationships from pure text, and the information therein is mainly presented in the format of natural language text. When it comes to the customer trade background review work, only being used for text data information extraction may lead to the loss of data information. Existing work shows that adding visual modal information can play an important role in information extraction work. Therefore, in this embodiment, in the context of big data and deep learning, the model automatically identifies multimodal customer trade background data and extracts the associated relationship of trade data from it, changing the logic of manually reviewing trade data, providing an efficient and convenient data retrieval method for the personnel of the handling bank, and improving the review efficiency.

[0065] In the embodiment of this application, the trade background information extraction model is used to review the trade background data combining text, images, and videos, extract the associated information between the pictures and texts in the provided materials, avoid data loss, and free the personnel from redundant files, only focusing on important information, which helps to improve the work efficiency and review accuracy of the personnel of the handling bank.

[0066] Embodiment 2

[0067] Figure 2The flowchart of a trade background review method provided in the second embodiment of the present invention. This embodiment is based on the above embodiment. After determining the trade background review result for the customer according to the score value, it further includes: verifying the trade background review result. As Figure 2 shown, the method includes:

[0068] Step S201, obtaining multimodal trade background data of the customer.

[0069] Optionally, obtaining multimodal trade background data of the customer includes: receiving a trade background review request and opening a multimodal data receiving port according to the trade background review request, where the multimodal data receiving port includes a text port, an image port, and a video port; receiving text trade background data through the text port, where the text trade background data includes a transaction contract, a copy of a bill of exchange, and a value-added tax invoice; receiving image trade background data through the image port, where the image trade background data includes a business license, product pictures, and site pictures; receiving video trade background data through the video port, where the video trade background data includes a product production process video and a product usage instruction video.

[0070] Optionally, before extracting different parameters from the trade background data by the trade background information extraction model according to the modality, it further includes: extracting key frames from the video trade background data to obtain image trade background data; performing image preprocessing operations on the image trade background data to obtain preprocessed image trade background data, where the image preprocessing operations include image cropping and image enhancement; performing text preprocessing operations on the text trade background data to obtain preprocessed text trade background data, where the text preprocessing operations include special character removal and word segmentation operations.

[0071] Step S202, extracting different parameters from the trade background data by the trade background information extraction model according to the modality.

[0072] Optionally, extracting different parameters from the trade background data by the trade background information extraction model according to the modality includes: converting the preprocessed text trade background data into a text vector by the trade background information extraction model, and converting the preprocessed image trade background data into an image vector; performing self-attention weighted calculation on the text vector to obtain an updated text vector; performing feature extraction on the image vector to obtain an updated image vector, where the dimensions of the updated image vector and the updated text vector are the same; extracting parameters according to the updated text vector and the updated image vector, where the parameters include entities, objects, and link relationships.

[0073] Optionally, parameter extraction is performed based on the updated text vector and the updated image vector, including: fusing the updated text vector and the updated image vector to obtain a fused vector; extracting entities from the updated text vector, extracting objects from the updated image vector, and extracting link relationships from the fused vector.

[0074] Step S203: Score the extracted parameters and determine the trade background audit result for the customer based on the score value.

[0075] Optionally, scoring the extracted parameters includes: obtaining the weights corresponding to the entities, objects, and link relationships; performing a weighted calculation on the entities, objects, and link relationships based on the weights to obtain a score value.

[0076] Optionally, determining the trade background audit result for the customer based on the score value includes: determining whether the score value exceeds a preset threshold. If so, it is determined that the trade background data is true and the customer's trade background audit passes; otherwise, it is determined that the trade background data is false and the customer's trade background audit fails.

[0077] Step S204: Verify the trade background audit result.

[0078] Specifically, in this embodiment, after auditing the customer's multimodal trade background data to obtain the trade background audit result, the trade background audit result will also be verified. For example, an artificial method is used for auditing to obtain the trade background audit result, and the artificial audit result is compared with the automatic audit result. If the error exceeds a certain value, it indicates that the current automatic audit result for the multimodal trade background data is incorrect. And in this embodiment, verification is performed at regular intervals. For example, verification is performed once every week, and each time verification is performed, the multimodal trade background data of the same customer is manually audited three times, and automatically audited three times, and the artificial audit result and the automatic audit result are compared three times. If the errors in all three comparisons exceed a certain value, it indicates that the trade background information extraction model is incorrect and the model parameters need to be adjusted again.

[0079] Among them, when the verification of the trade background audit result fails, a verification failure prompt message will be generated and sent to the business handler to prompt the business handler to promptly repair or upgrade the trade background information extraction model to ensure the accuracy of the audit result.

[0080] It should be noted that when sending the verification failure prompt message to the business handler in this embodiment, it can be displayed on the human-computer interaction interface or sent to the mobile terminal of the business handler, so that the business handler can obtain the prompt message from the human-computer interaction interface in a timely manner when on duty, or can also obtain the prompt message remotely when off duty, so that a quick response can be made based on the obtained prompt message to achieve a quick repair of the trade background information extraction model. Of course, only examples are given in this embodiment, and the specific display method of the prompt message is not limited.

[0081] In the embodiment of the present application, the trade background data combining text, images and videos is audited through the trade background information extraction model, the associated information between the pictures and texts in the provided materials is extracted, the lack of data is avoided, and the personnel are liberated from redundant files and only focus on important information, which helps to improve the work efficiency and audit accuracy of the personnel in the handling bank.

[0082] Embodiment III

[0083] Figure 3 It is a schematic structural diagram of a trade background audit device provided in Embodiment III of the present invention. As Figure 3 shown, the device includes: a trade background data acquisition module 310, a parameter extraction module 320, and an audit module 330.

[0084] Among them, the trade background data acquisition module 310 is used to acquire the multi-modal trade background data of the customer, where the modalities include text, images and videos;

[0085] The parameter extraction module 320 is used to extract different parameters from the trade background data according to the modalities through the trade background information extraction model;

[0086] The audit module 330 is used to score the extracted parameters and determine the trade background audit result for the customer according to the score value.

[0087] Optionally, the trade background data acquisition module is used to receive a trade background audit request and open a multi-modal data receiving port according to the trade background audit request, where the multi-modal data receiving port includes a text port, an image port and a video port;

[0088] Receive text trade background data through the text port, where the text trade background data includes a transaction contract, a copy of the bill of exchange and a value-added tax invoice;

[0089] Receive image trade background data through the image port, where the image trade background data includes a business license, product pictures and site pictures;

[0090] Receive video trade background data through a video port, where the video trade background data includes a product production process video and a product usage instruction video.

[0091] Optionally, the device further includes a trade background information preprocessing module for extracting key frames from the video trade background data to obtain image trade background data;

[0092] Perform image preprocessing operations on the image trade background data to obtain preprocessed image trade background data, where the image preprocessing operations include image cropping and image enhancement;

[0093] Perform text preprocessing operations on the text trade background data to obtain preprocessed text trade background data, where the text preprocessing operations include special character removal and word segmentation operations.

[0094] Optionally, a parameter extraction module is used to convert the preprocessed text trade background data into a text vector and convert the preprocessed image trade background data into an image vector through a trade background information extraction model;

[0095] Perform self-attention weighted calculation on the text vector to obtain an updated text vector;

[0096] Perform feature extraction on the image vector to obtain an updated image vector, where the dimensions of the updated image vector and the updated text vector are the same;

[0097] Extract parameters based on the updated text vector and the updated image vector, where the parameters include entities, objects, and link relationships.

[0098] Optionally, the parameter extraction module is further used to fuse the updated text vector and the updated image vector to obtain a fused vector;

[0099] Extract entities from the updated text vector, extract objects from the updated image vector, and extract link relationships from the fused vector.

[0100] Optionally, the audit module includes a scoring unit for obtaining the weights corresponding to the entities, objects, and link relationships;

[0101] Perform weighted calculation on the entities, objects, and link relationships based on the weights to obtain a score value.

[0102] Optionally, the audit module further includes an audit unit for determining whether the score value exceeds a preset threshold. If so, it is determined that the trade background data is true and the customer trade background audit is passed,

[0103] Otherwise, it is determined that the trade background data is false and the customer trade background audit fails.

[0104] The trade background audit device provided by the embodiments of the present invention can execute the trade background audit method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0105] Embodiment 4

[0106] Figure 4 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0107] As Figure 4 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0108] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0109] Processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as the trade background auditing method.

[0110] In some embodiments, the application to the trade background auditing method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the trade background auditing method described above may be executed. Alternatively, in other embodiments, processor 11 may be configured to execute the trade background auditing method by any other suitable means (e.g., by means of firmware).

[0111] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0112] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer programs are executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0113] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0114] To provide for interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0115] The systems and techniques described herein can be implemented in a computing system that includes backend components (such as, for example, a data server), or a computing system that includes middleware components (such as, for example, an application server), or a computing system that includes frontend components (such as, for example, a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (such as, for example, a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0116] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0117] Embodiment Five

[0118] The embodiment of the present invention also provides a computer program product, including a computer program which, when executed by a processor, implements the trade background auditing method provided in any embodiment of the present application.

[0119] In the process of implementing the computer program product, the computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0120] It should be noted that in the embodiments of the present application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned. They should be considered exemplary. The purpose is only to illustrate the feasibility in the implementation of the technical solution of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0121] Note that the above is only the preferred embodiment of the present invention and the applied technical principle. Those skilled in the art will understand that the present invention is not limited to the specific embodiments here. Various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A trade background review method, characterized in that: include: Acquire multimodal trade background data of the customer, wherein the multimodal trade background data includes text trade background data, image trade background data and video trade background data; Extracting different parameters from the trade background data according to the modality through a trade background information extraction model; Scoring is performed on the extracted parameters, and a trade background review result for the customer is determined based on the scoring value.

2. The method according to claim 1, characterized in that The obtaining of multi-modal trade background data of the customer includes: Receive a trade background review request, and open a multimodal data receiving port according to the trade background review request, wherein the multimodal data receiving port includes a text port, an image port, and a video port; Receiving the text trade background data through the text port, wherein the text trade background data includes a transaction contract, a copy of a bill of exchange, and a value-added tax invoice; Receiving the image trade background data through the image port, wherein the image trade background data includes a business license, product pictures and site pictures; The video trade background data is received through the video port, wherein the video trade background data includes a product production process video and a product use instruction video.

3. The method according to claim 1, characterized in that Before extracting different parameters from the trade background data according to the modality by using the trade background information extraction model, the method further includes: Extracting key frames from the video trade background data to obtain image trade background data; Performing an image preprocessing operation on the image trade background data to obtain preprocessed image trade background data, wherein the image preprocessing operation includes image cropping and image enhancement; A text preprocessing operation is performed on the text trade background data to obtain preprocessed text trade background data, wherein the text preprocessing operation includes special character removal and word segmentation operations.

4. The method according to claim 3, characterized in that The extracting different parameters from the trade background data according to the modality by using the trade background information extraction model includes: The preprocessed text trade background data is converted into a text vector and the preprocessed image trade background data is converted into an image vector by using the trade background information extraction model; Performing self-attention weighted calculation on the text vector to obtain an updated text vector; Performing feature extraction on the image vector to obtain an updated image vector, wherein the updated image vector and the updated text vector have the same dimension; Parameter extraction is performed according to the updated text vector and the updated image vector, wherein the parameters include entities, objects and link relationships.

5. The method according to claim 4, characterized in that The extracting parameters according to the updated text vector and the updated image vector comprises: Fusing the updated text vector and the updated image vector to obtain a fused vector; Entities are extracted from the updated text vector, objects are extracted from the updated image vector, and link relationships are extracted from the fused vector.

6. The method according to claim 4, characterized in that The step of scoring the extracted parameters comprises: Obtaining weights corresponding to the entity, the object, and the link relationship; The entity, the object and the link relationship are weightedly calculated based on the weight to obtain the score value.

7. The method according to claim 6, characterized in that Determining the trade background review result for the customer according to the score value includes: Determine whether the score value exceeds a preset threshold. If so, determine that the trade background data is true and the customer trade background review has passed. Otherwise, it is determined that the trade background data is false and the customer's trade background review fails.

8. A trade background audit device, characterized in that: The device comprises: A trade background data acquisition module, used to acquire multi-modal trade background data of a client, wherein the modalities include text, image and video; A parameter extraction module, used for extracting different parameters from the trade background data according to the modality through a trade background information extraction model; The audit module is used to score the extracted parameters and determine the trade background audit result of the customer according to the score value.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 7 when executed.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images