Procurement kickback risk warning and management method and system based on multimodal intelligent audit, electronic device, and computer-readable storage medium

Abstract

This invention, at the intersection of computer technology and business management, discloses a procurement kickback risk warning and management method and system based on multimodal intelligent auditing, as well as electronic devices and computer-readable storage media. The method includes the following steps: segmenting PDF invoice images using an improved Apache PDFBox parser; combining the CLIP model with OCR technology, using a cross-modal attention layer to correct the OCR text, generating multimodal consistency difference values, and determining the authenticity of the invoice; generating standardized risk events when false invoices are detected, extracting supplier, purchaser, and transaction edge data from a graph database to construct a time-series subgraph; identifying abnormal patterns based on a graph neural network, validating the risks using a time-series risk scoring model, and implementing a remediation strategy. Through multimodal fusion, dynamic graph analysis, and time-series modeling, the present invention addresses the problems of low false invoice detection accuracy, time-consuming abnormal fund tracing, and delayed risk prediction in traditional audits.

Description

Technical Field

[0001] The present invention belongs to the intersection of computer technology and business management, and specifically relates to a procurement kickback risk warning and management method and system based on multimodal intelligent auditing, an electronic device, and a computer-readable storage medium. Background Art

[0002] As corporate procurement scale expands, covert profiteering behaviors such as kickbacks are becoming more technical and covert, making traditional auditing methods incapable of addressing these new fraud patterns. Existing technologies have the following limitations:

[0003] 1. Shortcomings of Manual Auditing and Rule Engines

[0004] Traditional methods rely on manual review of paper contracts and invoices, supplemented by automated tools based on fixed rules (such as amount thresholds and supplier blacklists). For example, enterprise resource planning (ERP) systems use preset rules to screen out abnormal transactions (such as contract amounts exceeding the limit), but they have significant flaws: (1) High missed detection rate: Manual review misses over 46% of tampered contracts and false invoices; (2) Rigid rules: Static rules cannot adapt to complex tampering methods (such as adversarial sample attacks), and the update cycle takes several weeks.

[0005] 2. Limitations of Unimodal OCR Technology

[0006] Existing OCR technologies (such as Tesseract and Adobe PDF Extract) only extract text information and lack the ability to verify image semantics;

[0007] 3. The flaws of static transaction network analysis

[0008] Transaction analysis based on graph databases (such as Neo4j) often uses static rules, such as marking buyers with more than 10 transactions per month or suppliers with annual transactions exceeding their budget.

[0009] To address these issues, a technical solution is urgently needed that integrates multimodal data analysis, dynamic graph computing, and intelligent decision-making to achieve accurate early warning and closed-loop management of procurement kickback risks. This invention overcomes the limitations of traditional methods through the innovative design of CLIP-OCR multimodal correction, time-series subgraph construction, and dynamic scoring models, significantly improving audit efficiency and accuracy. Summary of the Invention

[0010] The purpose of the present invention is to propose a procurement kickback risk warning and management method and system based on multimodal intelligent auditing, electronic equipment, and computer-readable storage medium to solve the problems of low accuracy in false invoice detection, long time-consuming abnormal fund tracing, and delayed risk prediction in traditional auditing.

[0011] The present invention provides a procurement kickback risk warning and management method based on multimodal intelligent auditing, which includes the following steps:

[0012] Step 1: Input the PDF invoice image to be inspected and segment it using the improved Apache PDFBox parser to separate the directly editable text blocks and image blocks. Apply OCR technology to the image blocks to generate the image embedded text set {T} and the corresponding original image block set {I}.

[0013] Step 2: Input the image embedded text set {T} into the text encoder in the CLIP model to generate the text feature vector CLIP text (T); Input the original image block set {I} into the image encoder in the CLIP model to generate the image feature vector CLIP image (I) Using the cross-modal attention layer in the CLIP model to calculate the association weights between text and image features , dynamically correct OCR extraction results; based on the association weight , calculate the corrected text set ;

[0014] Step 3: Calculate the corrected text set The difference value with the image block set {I} , by comparing with the preset threshold Compare and check the authenticity of the invoice.

[0015] Step 4: When an invoice is determined to be fraudulent, a standardized risk event is generated. Based on the supplier_id and purchaser_id fields in the risk event, the supplier node, purchaser node, and transaction edge data are extracted from the graph database. This data is then used to construct a time-series subgraph.

[0016] Step 5: Based on the time-series subgraph constructed above, use the graph neural network (GNN) to perform preliminary detection to generate a list of high-risk nodes and abnormal pattern labels; then verify the high-risk nodes using the time-series risk scoring model, and perform corresponding disposal operations based on the verification results.

[0017] As described above, a procurement kickback risk warning and management method based on multimodal intelligent auditing is

[0018] The image embedded text set {T} in step 1 is a set of texts recognized from image blocks using OCR technology.

[0019] As described above, a procurement kickback risk warning and management method based on multimodal intelligent auditing is

[0020] The association weight in step 2 The calculation formula is:

[0021] ,

[0022] Among them, d is the feature dimension; is the transpose operation;

[0023] The association weight , calculate the corrected text set , the calculation formula is: , where T is the set of text embedded in the image; is a text decoder; is the image feature vector.

[0024] As described above, a procurement kickback risk warning and management method based on multimodal intelligent auditing is

[0025] The difference between the corrected text set and the image block set in step 3 The specific calculation formula is:

[0026] ,

[0027] in, is the L2 norm;

[0028] The threshold The default setting is 0.12, which can be adaptively adjusted according to the supplier's credit score. The specific adjustment formula is:

[0029] .

[0030] As described above, a procurement kickback risk warning and management method based on multimodal intelligent auditing is

[0031] The step 4 is specifically as follows:

[0032] Step 4.1: When When the invoice is false, it is judged as a false invoice, triggering the generation of a standardized event;

[0033] Step 4.2: Generate standardized risk events. Standardized risk events include the following core fields: event_id, supplier_id, purchaser_id, invoice_id, timestamp, and risk_level.

[0034] Step 4.3: Push the data to the graph neural network (GNN) in real time through the message queue.

[0035] Step 4.4: Based on the supplier_id and purchaser_id in the risk event, use the nGQL query language to extract the supplier node, purchaser node, and transaction edge data information from the graph database, and build a time-series subgraph based on the above data information.

[0036] As described above, a procurement kickback risk warning and management method based on multimodal intelligent auditing is

[0037] The preliminary detection using the graph neural network GNN in step 5 includes using the temporal graph attention network T-GAT to perform high-frequency trading detection and procurement kickback identification; wherein, high-frequency trading detection means that if the transaction edge weight between a supplier and a purchaser is significantly higher than the historical average in a short period of time, it is marked as "high-frequency trading risk"; and when detecting a closed-loop path, the sum of the path weights is calculated, and if it exceeds the threshold, it is marked as "procurement kickback".

[0038] As described above, a procurement kickback risk warning and management method based on multimodal intelligent auditing is

[0039] When verifying high-risk nodes using the timing risk scoring model in step 5, the specific formula for timing risk scoring is:

[0040] Where n is the number of transactions; The default value is 0.1, which represents the risk score time decay coefficient; The transaction time interval in days.

[0041] The present invention also discloses a procurement rebate risk warning and management based on multimodal intelligent auditing

[0042] A system comprising:

[0043] Image and text segmentation and extraction module: This module is used to segment the PDF invoice image to be inspected using an improved Apache PDFBox parser, separating directly editable text blocks and image blocks. It also applies OCR technology to the image blocks to generate an embedded text set {T} and a corresponding original image block set {I}.

[0044] CLIP module: used to use the text encoder in the CLIP module to encode the image embedded text set {T} to generate the text feature vector CLIP text (T); Use the image encoder in the CLIP module to encode the original image block set {I} to generate the image feature vector CLIP image (I) Using the cross-modal attention layer in the CLIP model to calculate the association weights between text and image features , dynamically correct OCR extraction results; based on the association weight , calculate the corrected text set ;

[0045] Image-text consistency determination module: used to calculate the corrected text set The difference value with the image block set {I} , by comparing with the preset threshold Compare and check the authenticity of invoices;

[0046] The time-series subgraph construction module is used to generate a standardized risk event when an invoice is determined to be fraudulent. Based on the supplier_id and purchaser_id fields in the risk event, the module extracts the supplier node, purchaser node, and transaction edge data from the graph database. This data is then used to construct a time-series subgraph.

[0047] Risk grading and disposal module: It is used to perform preliminary detection based on the time-series subgraph constructed above using the graph neural network (GNN) to generate a list of high-risk nodes and abnormal pattern labels; then, the high-risk nodes are verified using the time-series risk scoring model, and corresponding disposal operations are performed based on the verification results.

[0048] The present invention also discloses an electronic device, comprising one or more processors; a storage device for storing one or more computer programs. When the one or more computer programs are executed by the one or more processors, the electronic device implements the above-mentioned procurement kickback risk warning and management method based on multimodal intelligent auditing.

[0049] The present invention also discloses a computer-readable storage medium, characterized in that a computer program is stored thereon. When the computer program is executed by a processor of an electronic device, the electronic device executes the above-mentioned procurement rebate risk warning and management method based on multimodal intelligent auditing. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 This is a flowchart of the steps of the procurement kickback risk warning and management method based on multimodal intelligent auditing. DETAILED DESCRIPTION

[0052] The present invention will be described in detail below through examples.

[0053] See also Figure 1 , which shows a flowchart of the steps of a procurement kickback risk warning and management method based on multimodal intelligent auditing provided by an embodiment of the present invention, such as Figure 1 As shown, the procurement kickback risk warning and management method based on multimodal intelligent auditing provided by the embodiment of the present application includes the following steps 1 to 5:

[0054] Step 1: Input the PDF invoice image to be inspected and segment it using the improved Apache PDFBox parser to separate the directly editable text blocks and image blocks. Apply OCR technology to the image blocks to generate the image embedded text set {T} and the corresponding original image block set {I}.

[0055] It should be noted that the improved design of the Apache PDFBox parser is as follows: (1) PDF files are divided into independent tasks by page, and the parsing speed is increased to at least 300 pages per minute through multi-threaded concurrent processing; (2) common font libraries are preloaded into memory when the parser starts, reducing rendering time by more than 20%; (3) an image-text alignment algorithm is used, based on the PDFTextStripperByArea class of Apache PDFBox, to improve the area segmentation logic, and through adaptive grid division (grid density is dynamically adjusted, and the error tolerance is ≤5px), directly editable text blocks T and their corresponding image blocks I are accurately extracted. Among them, (1) directly editable text blocks: extracted directly from the PDF text layer by PDFBox, do not participate in multimodal verification, and are directly used for business logic (such as contract amount comparison); (2) image embedded text set {T}: refers only to the text recognized from the image blocks by OCR; (3) image block set {I}: retains the original image data, which refers to the set of image blocks obtained by segmenting the PDF invoice image through the improved Apache PDFBox parser, for the CLIP image encoder to extract visual features.

[0056] Step 2: Input the image embedded text set {T} into the text encoder in the CLIP model to generate the text feature vector CLIP text (T); Input the original image block set {I} into the image encoder in the CLIP model to generate the image feature vector CLIP image (I) Using the cross-modal attention layer in the CLIP model to calculate the association weights between text and image features , dynamically correct OCR extraction results; based on the association weight , calculate the corrected text set ;

[0057] It should be noted that the cross-modal attention layer in the CLIP model is used to calculate the association weights of text and image features. , and its calculation formula is:

[0058] , where d is the feature dimension; is the transpose operation;

[0059] In the above formula, the core is to assign weights by calculating the similarity between features; in particular, by using the text feature vector and image feature vector The dot product operation of the transpose is used to measure the semantic relevance between text features and image features. The larger the dot product value, the higher the consistency between the two. In order to avoid the dot product value being too large when the dimension d is large, we divide it by Control gradient stability; transpose operation is to achieve dimension matching between vectors; and Softmax normalization is to convert the dot product result into probability distribution to ensure weight , so as to reasonably distribute the contribution ratio of text and image features. The physical meaning of this weight is the credibility weight of the text feature. For example, if the image feature is highly consistent with the text feature (such as the invoice is clearly printed and the OCR extraction is accurate), then , mainly based on the OCR result; if the image features are significantly different from the text features (such as the text is tampered with but the image background remains original), then , mainly based on image decoding results.

[0060] Based on association weight , calculate the corrected text set , and its calculation formula is:

[0061] , where T is the set of text embedded in the image; is a text decoder; is the image feature vector.

[0062] In this formula, Z-score standardization can be used to make its mean 0 and variance 1, eliminating dimensional differences; and α and (1−α) are used as complementary weights to ensure that the weighted features are still within the distribution range of the original features.

[0063] and This is not the native inverse model of CLIP, but an independently trained fully connected network with the following structure: Input: CLIP image features (dimension d=512); Hidden layer: 2-layer MLP (dimension 512→256→512), with GELU activation function; Output: decoded text features, with the same dimension as the OCR features.

[0064] It can be seen that in this step 2, the multimodal fusion of the CLIP model and OCR technology, through two-way feature interaction, breaks through the isolated application mode of traditional OCR and CLIP, and achieves the following technical effects: using CLIP's semantic understanding ability to correct OCR output and resist adversarial attacks; through cross-modal attention, deep fusion of image and text features is achieved, and hidden fraud of "text tampering but image retention" is accurately identified.

[0065] Step 3: Calculate the corrected text set The difference value with the image block set {I} , by comparing with the preset threshold Compare and check the authenticity of invoices;

[0066] It should be noted that the difference between the corrected text set and the image block set is The calculation formula is: ,in, is the L2 norm (Euclidean distance). The smaller its value, the higher the semantic consistency between the text and the image. The threshold Δ is preset to 0.12 by default and can be adaptively adjusted dynamically according to the supplier's credit score. The specific adjustment formula is:

[0067]

[0068] By dynamically adjusting the Δ value, the threshold is relaxed for high-credit suppliers to reduce false positives, while the threshold is tightened for low-credit suppliers to improve the detection rate. For example, when supplier A has a credit score of 70, the threshold is adjusted to Δ=0.12 1.2=0.144, to prevent high-credit suppliers from being misjudged due to minor formatting issues; when supplier B has a credit score of 30 points, the threshold is adjusted to Δ=0.12 0.8=0.096, the threshold is tightened, improving the detection rate;

[0069] It can be seen that in this step 3, By quantifying the semantic differences between text and image features, a multimodal fusion detection method is provided for procurement kickback audits, achieving the following technical effects: compensating for the single-modal defects of traditional OCR and realizing more comprehensive anti-tampering capabilities; dynamic thresholds are linked with credit scores to balance risk management and business efficiency.

[0070] Step 4: When an invoice is determined to be fraudulent, a standardized risk event is generated. Based on the supplier_id and purchaser_id fields in the risk event, the supplier node, purchaser node, and transaction edge data are extracted from the graph database. This data is then used to construct a time-series subgraph.

[0071] It should be noted that: the graph database, as a persistent storage system, is responsible for storing the original data of all static nodes (suppliers, buyers) and edges (transaction records), and providing basic query functions; the time-series subgraph is a dynamically generated temporary data structure for specific analysis targets (such as high-risk events). It is built based on part of the data in the graph database and is used for real-time or near real-time analysis.

[0072] Standardized risk event generation and data extraction:

[0073] Fake invoice determination and event generation:

[0074] 1. Trigger condition: When the difference value When >Δ, it is determined to be a false invoice.

[0075] 2. Event structure: Generates standardized risk events, including the following core fields:

[0076] {

[0077] "event_id": "RISK_001",

[0078] "supplier_id": "S_123",

[0079] "purchaser_id": "P_456",

[0080] "invoice_id": "INV_789",

[0081] "timestamp": "2023-10-05T14:30:00Z",

[0082] "risk_level": "high"

[0083] }

[0084] 3. Event transmission: Pushed to the graph neural network (GNN) module in real time through the message queue.

[0085] Graph database data extraction:

[0086] 1. Query target: Extract the following data based on the supplier_id and purchaser_id in the event:

[0087] Supplier node: contains attributes such as supplier ID, credit score, number of historical risk events, registration information, etc.

[0088] Purchaser node: contains attributes such as purchaser ID, approval authority, job rank, number of related contracts, etc.

[0089] Bank account node (optional): If you need to analyze the fund flow path, it can include attributes such as account ID, bank account, and balance changes;

[0090] Transaction side: All transaction records within a limited time window (e.g., the last three months), including amount, timestamp, contract number, etc.

[0091] Funds flow edge (optional): If a bank account node is included, record the flow of funds (e.g., buyer → account → supplier).

[0092] 2. Query language: Use a graph database query language (such as Nebula Graph's nGQL);

[0093] 3. Output results: A data set containing supplier nodes, buyer nodes, and transaction edges is formed.

[0094] Time series subgraph construction and feature enhancement:

[0095] 1. Node and edge screening: Only supplier and buyer nodes and transaction edges related to risk events are retained;

[0096] 2. Time window constraint: The timestamp of the transaction edge must be within a preset window (e.g., the last three months);

[0097] 3. Dynamic weight calculation: Add a time-decay weight to each transaction edge:

[0098]

[0099] Where t is the transaction time interval, that is, the time difference between the current time and the transaction time, in days; is the graph network attenuation coefficient (default 0.1), which controls the weight attenuation speed of historical transactions and has a value range of [0.05, 0.2]. i represents the transaction initiator node, such as the supplier; j represents the transaction receiver node, such as the buyer.

[0100] In particular, for the time decay weight, as t increases, the weight decreases exponentially, which conforms to the business logic that "recent transactions are more important than long-term transactions"; and by adjusting , can flexibly control the speed of weight decay. For example: when =0.01 When , the decay is slower, retaining the impact of longer-term historical transactions; when =0.2 When the weight is too small or too large, it decays quickly, focusing only on recent transactions. Of course, to eliminate the problem of too small or too large weights, weights can be standardized or a lower limit can be set based on business rules to ensure that long-term transactions still retain a certain influence. These can be achieved through conventional settings.

[0101] Feature fusion and enhancement

[0102] 1. Node feature enhancement: Supplier node: Add the recent false invoice ratio (for example, two high-risk events were detected in the past month); Purchaser node: Add the total approved amount and the number of abnormal transaction marks.

[0103] 2. Edge feature expansion: Label the contract type (normal procurement / supplementary agreement), transaction status (completed / pending payment), and other tags.

[0104] As can be seen, in step 4, the construction of the time-series subgraph relies on supplier nodes, buyer nodes, and transaction edges extracted from the graph database. However, through dynamic weight calculation, time window screening, and multi-feature fusion, a detailed modeling of transaction behavior is achieved. This approach not only ensures transparency of data sources but also significantly improves the accuracy and efficiency of risk detection by quantifying time decay effects and abnormal patterns.

[0105] Step 5: Based on the time-series subgraph constructed above, a preliminary detection is performed using the graph neural network (GNN) to generate a list of high-risk nodes and abnormal pattern labels. The high-risk nodes are then verified using the time-series risk scoring model, and corresponding disposal operations are performed based on the verified risk level.

[0106] It should be noted that: First, when designing the GNN model, the temporal graph attention network (T-GAT) is used, and the input features include: node features: credit score, number of risk events, and approval authority level; edge features: transaction amount, time decay weight, and contract type label.

[0107] Abnormal pattern detection includes: high-frequency trading detection and procurement kickback identification, specifically:

[0108] If the transaction weight between a supplier and a buyer is significantly higher than the historical average in a short period of time, it is marked as "high-frequency trading risk";

[0109] Detect closed-loop paths (e.g., supplier → buyer → associated account → supplier), calculate the total path weight, and if it exceeds the threshold, mark it as "purchase rebate." The specific process is as follows:

[0110] Since the weight of each edge is determined by the transaction amount and the time decay coefficient , the calculation example of the sum of path weights is:

[0111] Assume that the current time is 2023-10-08, and the three edge information of a closed-loop path are as follows:

[0112] 1. Transaction side: Amount 50,000 yuan, time difference 7 days, corresponding w1 ≈ 24850;

[0113] 2. Transfer side (Purchaser → Account): Amount 50,000 yuan, time difference 7 days, corresponding w2 ≈ 24850;

[0114] 3. Transfer side (account → supplier): Amount 50,000 yuan, time difference 7 days, corresponding w3 ≈ 24850;

[0115] Then the total path weight is: Total Weight = w1 + w2 + w3 = 74,550;

[0116] Finally, by comparing the sum of the path weights with the preset threshold, it is determined whether there is suspected procurement kickback.

[0117] The initial detection output is: generating a list of high-risk nodes and abnormal pattern labels (such as "high-frequency trading" and "purchase kickbacks"); an example is:

[0118] {

[0119] "supplier_id": "S_123",

[0120] "purchaser_id": "P_456",

[0121] "risk_type": "high_frequency_transaction",

[0122] "confidence": 0.92

[0123] }

[0124] Then, the specific operations for verification using the time series risk scoring model are as follows:

[0125] 1. Input data: high-risk nodes output by GNN and their associated transaction records;

[0126] 2. Calculate the corresponding timing risk score. The specific formula is:

[0127]

[0128] Where n is the number of transactions; The default value is 0.1, which represents the risk score time decay coefficient, and the value range is [0.05, 0.3]; The transaction time interval in days.

[0129] For example, supplier S_123 has five transactions in the past three months, with transaction amount / contract amount ratios of 1.2, 0.8, 1.5, 1.1, and 1.3, respectively, and time intervals of 7, 15, 30, 60, and 90 days:

[0130] E risk =1.2×e -0.7 +0.8×e -1.5 +1.5×e -3 +1.1×e -6 +1.3×e -9 ≈8.6

[0131] 3. Based on the obtained time series analysis score, compare it with the preset threshold, classify the risk level, and

[0132] Execute appropriate actions;

[0133] Assuming the preset threshold is 5.0, if E risk > 5.0, then the GNN detection result is confirmed to be valid;

[0134] (1) Graded response:

[0135] when E risk 8.0 is high risk: freeze the contract immediately and initiate an internal investigation;

[0136] When 5.0 E risk < 8.0 is medium risk: bidding authority is restricted and submission of explanatory materials is required;

[0137] when E risk <5 .0 is low risk: mark as observation object and review regularly.

[0138] (2) Automatically execute corresponding level of disposal measures, including but not limited to: contract freezing, authority demotion, account monitoring, credit score update, etc.

[0139] Therefore, in step 5, the present invention first performs preliminary detection through a graph neural network and then verifies it through a temporal risk scoring model. The former captures implicit associations from topological relationships, and the latter quantifies time-sensitive risks. The two-dimensional cross-talk reduces the false alarm rate.

[0140] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and purpose of the present invention.

Claims

1. A procurement kickback risk warning and management method based on multimodal intelligent auditing, characterized by: The method comprises the following steps: Step 1: Input the PDF invoice image to be inspected and segment it using the improved Apache PDFBox parser to separate the directly editable text blocks and image blocks. Apply OCR technology to the image blocks to generate the image embedded text set {T} and the corresponding original image block set {I}. Step 2: Input the image embedded text set {T} into the text encoder in the CLIP model to generate the text feature vector CLIP text (T); Input the original image block set {I} into the image encoder in the CLIP model to generate the image feature vector CLIP image (I) Using the cross-modal attention layer in the CLIP model to calculate the association weights between text and image features , dynamically correct OCR extraction results; based on the association weight , calculate the corrected text set ; The association weight The calculation formula is: ， Among them, d is the feature dimension; is the transpose operation; The association weight , calculate the corrected text set , the calculation formula is: , where T is the set of text embedded in the image; is a text decoder; is the image feature vector; Step 3: Calculate the corrected text set The difference value with the image block set {I} , by comparing with the preset threshold Compare and check the authenticity of invoices; Step 4: When an invoice is determined to be fraudulent, a standardized risk event is generated. Based on the supplier_id and purchaser_id fields in the risk event, the supplier node, purchaser node, and transaction edge data are extracted from the graph database. This data is then used to construct a time-series subgraph. Step 5: Based on the time-series subgraph constructed above, use the graph neural network (GNN) to perform preliminary detection to generate a list of high-risk nodes and abnormal pattern labels; then verify the high-risk nodes using the time-series risk scoring model, and perform corresponding disposal operations based on the verification results.

2. The procurement kickback risk warning and management method based on multimodal intelligent auditing according to claim 1 is characterized in that: The image embedded text set {T} in step 1 is a set of texts recognized from image blocks using OCR technology.

3. The procurement kickback risk warning and management method based on multimodal intelligent auditing according to claim 2 is characterized in that: The difference between the corrected text set and the image block set in step 3 The specific calculation formula is: in, is the L2 norm; The threshold The default setting is 0.12, which can be adaptively adjusted according to the supplier's credit score. The specific adjustment formula is: 。 4. The procurement kickback risk warning and management method based on multimodal intelligent auditing according to claim 3 is characterized in that: The step 4 is specifically as follows: Step 4.1: When When >Δ, it is determined to be a fake invoice, triggering the generation of a standardized event; Step 4.2: Generate standardized risk events. Standardized risk events include the following core fields: event_id, supplier_id, purchaser_id, invoice_id, timestamp, and risk_level. Step 4.3: Push the data to the graph neural network (GNN) in real time through the message queue. Step 4.4: Based on the supplier_id and purchaser_id in the risk event, use the nGQL query language to extract the supplier node, purchaser node, and transaction edge data information from the graph database, and build a time-series subgraph based on the above data information.

5. The procurement kickback risk warning and management method based on multimodal intelligent auditing according to claim 4 is characterized in that: The preliminary detection using the graph neural network (GNN) in step 5 includes using the temporal graph attention network (T-GAT) to perform high-frequency trading detection and purchase rebate identification; wherein, high-frequency trading detection means that if the transaction edge weight between a supplier and a purchaser is significantly higher than the historical average in a preset time window, it is marked as "high-frequency trading risk"; and when detecting a closed-loop path, the sum of the path weights is calculated, and if it exceeds the threshold, it is marked as "purchase rebate".

6. The procurement kickback risk warning and management method based on multimodal intelligent auditing according to claim 5 is characterized in that: When verifying high-risk nodes using the timing risk scoring model in step 5, the specific formula for timing risk scoring is: Where n is the number of transactions; The default value is 0.1, which represents the risk score time decay coefficient; The transaction time interval in days.

7. A procurement kickback risk warning and management system based on multimodal intelligent auditing, characterized by: The system comprises: Image and text segmentation and extraction module: This module is used to segment the PDF invoice image to be inspected using an improved Apache PDFBox parser, separating directly editable text blocks and image blocks. It also applies OCR technology to the image blocks to generate an embedded text set {T} and a corresponding original image block set {I}. CLIP module: used to use the text encoder in the CLIP module to encode the image embedded text set {T} to generate the text feature vector CLIP text (T); Use the image encoder in the CLIP module to encode the original image block set {I} to generate the image feature vector CLIP image (I) Using the cross-modal attention layer in the CLIP model to calculate the association weights between text and image features , dynamically correct OCR extraction results; based on the association weight , calculate the corrected text set ; The association weight The calculation formula is: ， Among them, d is the feature dimension; is the transpose operation; The association weight , calculate the corrected text set , the calculation formula is: , where T is the set of text embedded in the image; is a text decoder; is the image feature vector; Image-text consistency determination module: used to calculate the corrected text set The difference value with the image block set {I} , by comparing with the preset threshold Compare and check the authenticity of invoices; The time-series subgraph construction module is used to generate a standardized risk event when an invoice is determined to be fraudulent. Based on the supplier_id and purchaser_id fields in the risk event, the module extracts the supplier node, purchaser node, and transaction edge data from the graph database. This data is then used to construct a time-series subgraph. Risk grading and disposal module: It is used to perform preliminary detection based on the time-series subgraph constructed above using the graph neural network (GNN) to generate a list of high-risk nodes and abnormal pattern labels; then, the high-risk nodes are verified using the time-series risk scoring model, and corresponding disposal operations are performed based on the verification results.

8. An electronic device, characterized in that: It includes one or more processors; a storage device for storing one or more computer programs. When the one or more computer programs are executed by the one or more processors, the electronic device implements the procurement rebate risk warning and management method based on multimodal intelligent auditing as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of an electronic device, the electronic device executes the procurement kickback risk warning and management method based on multimodal intelligent auditing as described in any one of claims 1-6.

Images