Risk data identification method and device, equipment and medium

Abstract

The invention provides a risk data identification method and device, equipment and a medium, relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, NLP, agents, large models and the like, and can be applied to the fields of finance, finance and the like. According to the implementation scheme, the method comprises the steps of obtaining a plurality of to-be-recognized receipts and detail information associated with the to-be-recognized receipts; according to the detail information of any to-be-identified document, determining a risk score of any to-be-identified document; according to at least one set business rule sent by the client and the risk score of each document to be identified, identifying a first risk document from the plurality of documents to be identified; and the risk data is identified from the detail information of the first risk document, and the risk data is sent to the client, so that the interpretability of risk data identification is improved, the error processing of the document is reduced, the financial management efficiency is improved, the financial control and risk management capabilities of an enterprise are enhanced, meanwhile, the user demand is met, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the technical fields of deep learning, natural language processing (NLP), intelligent agents and large models, which can be applied to fields such as finance and finance, and in particular to a method, device, equipment and medium for identifying risk data. Background Art

[0002] With the expansion of enterprise scale and business scope, financial management has become the core link in enterprise management. As an important part of the enterprise's financial flow, the accuracy and compliance of various documents are directly related to the enterprise's financial security, tax compliance and operational efficiency. In order to ensure the financial security of the enterprise, improve tax compliance and optimize operational efficiency, it is necessary to accurately identify the risk data hidden in various documents. Therefore, how to accurately identify the risk data in various documents is very important. Summary of the invention

[0003] The present disclosure provides a method, apparatus, device and medium for identifying risk data.

[0004] According to one aspect of the present disclosure, a method for identifying risk data is provided, comprising:

[0005] Acquire multiple documents to be identified and detailed information associated with each of the documents to be identified;

[0006] For any document to be identified, determine the risk score of the document to be identified based on the detailed information of the document to be identified;

[0007] Identifying a first risk document from the multiple documents to be identified according to at least one set business rule sent by the client and the risk score of each of the documents to be identified;

[0008] Risk data is identified from the detailed information of the first risk document, and the risk data is sent to the client.

[0009] According to another aspect of the present disclosure, there is provided a risk data identification device, comprising:

[0010] An acquisition module, used for acquiring a plurality of documents to be identified and detailed information associated with each of the documents to be identified;

[0011] A determination module, configured to determine, for any document to be identified, a risk score of the document to be identified based on detailed information of the document to be identified;

[0012] A first identification module, configured to identify a first risk document from the multiple documents to be identified according to at least one set business rule sent by the client and the risk score of each of the documents to be identified;

[0013] The second identification module is used to identify risk data from the detailed information of the first risk document and send the risk data to the client.

[0014] According to another aspect of the present disclosure, there is provided an electronic device, including:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method proposed in the above aspect of the present disclosure.

[0018] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium of computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method proposed in the above aspect of the present disclosure.

[0019] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the method proposed in the above aspect of the present disclosure when executed by a processor.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0022] Figure 1 A flowchart of a method for identifying risk data provided in the first embodiment of the present disclosure;

[0023] Figure 2 A flowchart of a method for identifying risk data provided in the second embodiment of the present disclosure;

[0024] Figure 3 A flowchart of a method for identifying risk data provided in Embodiment 3 of the present disclosure;

[0025] Figure 4 A flowchart of a method for identifying risk data provided in Embodiment 4 of the present disclosure;

[0026] Figure 5 A schematic diagram of a system architecture for implementing a risk data identification method provided by an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram of the structure of a risk data identification device provided in Embodiment 5 of the present disclosure;

[0028] Figure 7 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0029] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0030] At present, risk prediction of documents is mainly carried out through machine learning and deep learning technology, which includes the following steps:

[0031] 1. Data collection: Collect a large amount of financial documents (such as reimbursement documents), including normal documents and illegal documents;

[0032] 2. Data processing: perform text extraction, data cleaning, feature extraction and other operations on these documents;

[0033] 3. Model training: Build machine learning or deep learning algorithms, such as decision trees, support vector machines, neural networks, etc., to train the document risk scoring model. Documents with high risk scores are more likely to be illegal documents, and continuous optimization and iteration are performed;

[0034] 4. Model deployment: Apply the trained model to identify risks of new documents. The identification results are directly used in the financial audit process to help financial auditors quickly identify and process documents with high risk of violations.

[0035] The above machine learning and deep learning technologies can lead to the following problems when predicting the risk of documents:

[0036] 1. Poor recommendation capability: Financial auditors are unable to set business rules to control the system to push risk documents that meet existing business needs;

[0037] 2. Low interpretability: Since risk document scoring models generally use neural network architecture models, the model interpretability is low. At the same time, financial processing is a relatively sensitive link in the enterprise, so it is difficult for financial auditors to understand such models, and then it is difficult to gain the trust of financial auditors, which affects the utilization rate of the system;

[0038] 3. Poor user experience: high learning cost, relatively fixed page operations, and poor interactive experience.

[0039] In response to at least one of the above problems, the present disclosure proposes a method, device, equipment and medium for identifying risk data.

[0040] The following describes the risk data identification method, apparatus, device and medium of the embodiments of the present disclosure with reference to the accompanying drawings.

[0041] Figure 1 This is a flow chart of the risk data identification method provided in the first embodiment of the present disclosure.

[0042] The embodiment of the present disclosure takes the risk data identification method configured in a risk data identification device as an example. The risk data identification device can be applied to any electronic device so that the electronic device can perform the risk data identification function.

[0043] Among them, the electronic device can be any device with computing capabilities, such as a computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0044] It should be noted that the risk data identification method of the embodiment of the present disclosure can be executed by an intelligent agent based on a large model, wherein the intelligent agent refers to a computer program based on a large language model that has planning ability, memory ability, ability to use tool functions, and can autonomously complete a given task.

[0045] like Figure 1 As shown, the risk data identification method may include the following steps:

[0046] Step 101, obtaining a plurality of documents to be identified and detailed information associated with each document to be identified.

[0047] In the embodiment of the present disclosure, the document to be identified (eg, a reimbursement document) may be an electronic document sent by a client, or may be a document obtained by scanning a paper document.

[0048] In order to accurately obtain the detailed information of any document to be identified, in the embodiment of the present disclosure, the document to be identified can be identified and queried to obtain the detailed information associated with each document to be identified, wherein the detailed information includes but is not limited to the document information and associated information corresponding to the document to be identified. The document information of the document to be identified can be obtained by performing OCR recognition on the document, and the associated information of the document to be identified can be obtained by query. In order to improve the comprehensiveness of the detailed information of the document to be identified, the associated information includes at least one of the following information: document object information, document processing record information, document audit record information and document review record information.

[0049] Step 102: for any document to be identified, determine the risk score of any document to be identified according to the detailed information of any document to be identified.

[0050] In the embodiment of the present disclosure, the similarity between any document to be identified and the corresponding historical violation document is calculated, and the risk score of any document to be identified is determined based on the similarity, wherein the risk score of any document to be identified and the corresponding similarity are positively correlated, that is, the higher the similarity between any document to be identified and the corresponding historical violation document, the higher the risk score of any document to be identified.

[0051] Step 103: Identify a first risk document from multiple documents to be identified based on at least one set business rule sent by the client and the risk score of each document to be identified.

[0052] In the disclosed embodiment, according to at least one set business criterion provided by the client, and in combination with the risk score obtained by evaluating each document to be identified, the document with the greatest potential risk is determined from multiple documents to be identified, and the document with the greatest potential risk is used as the first risk document. The set business rules may include: the minimum number of documents to be drawn for each document category (such as the expense category), the minimum proportion of key document categories to be drawn, the upper limit of the reimbursement amount, the restriction of expense types, etc.

[0053] Step 104: identify risk data from the detailed information of the first risk document, and send the risk data to the client.

[0054] In the disclosed embodiment, the detailed information of the first risk document is parsed, and based on the document verification criteria, risk data in the detailed information that does not conform to the document verification criteria is extracted, and then the risk data is sent to the client so that relevant personnel can further review the risk data; wherein, the risk data may be, for example: reimbursement item name, amount, invoice number, approver information, etc.

[0055] The risk data identification method of the disclosed embodiment calculates a risk score based on the detailed information of the document to be identified, and then identifies a first risk document with high risk that meets user needs from multiple documents to be identified based on set business rules and risk scores. Finally, the risk data is located from the detailed information of the first risk document, and the risk data is sent to the client, so that relevant personnel can understand the risk situation of the document more clearly, improves the interpretability of risk data identification, reduces document error processing, improves financial management efficiency, enhances the financial control and risk management capabilities of the enterprise, and at the same time meets user needs and improves user experience.

[0056] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, and are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0057] In order to clearly illustrate how any embodiment of the present disclosure identifies a first risky document from multiple documents to be identified based on at least one set business rule sent by a client and the risk score of each document to be identified, the present disclosure proposes a method for identifying risk data.

[0058] Figure 2 This is a flow chart of the risk data identification method provided in the second embodiment of the present disclosure.

[0059] like Figure 2 As shown, the risk data identification method may include the following steps:

[0060] Step 201, obtaining a plurality of documents to be identified and detailed information associated with each document to be identified.

[0061] Step 202: for any document to be identified, determine the risk score of any document to be identified according to the detailed information of any document to be identified.

[0062] Step 203: classify each document to be identified according to the target category of each document to be identified, so as to obtain documents under at least one target category.

[0063] In order to improve the accuracy of financial management, in the embodiment of the present disclosure, the target category is determined based on the core fields in the corresponding document to be identified. For example, the target category is determined based on the expense field in the document. The target category is the expense category. The target category includes but is not limited to: office supplies expenses, travel expenses, transportation expenses, catering expenses, training expenses, conference expenses, equipment purchase expenses, maintenance expenses, etc. The documents to be identified are classified according to their target categories. For example, the collected documents to be identified are grouped and divided according to the expense type indicated on the document to obtain documents under each expense category.

[0064] It should be noted that if a document involves multiple categories (for example, a reimbursement document includes both travel expenses and meal expenses), it may be necessary to split it or classify it according to the main category.

[0065] Step 204: sort the documents under any target category according to the risk scores of the documents under any target category to obtain a first sorting sequence.

[0066] In order to efficiently identify and prioritize high-risk documents, in the present disclosed embodiment, for the documents to be identified under any target category, the risk score of each document under the target category is obtained, and then the documents under the target category are sorted according to the risk score. For example, documents with higher risk scores may be placed at the front, and finally an ordered list, i.e., the first sorting sequence, is formed.

[0067] Step 205: Determine a first risk document from the first sorting sequence according to at least one set business rule.

[0068] In order to ensure that documents that are both high-risk and meet specific business requirements can be processed preferentially, based on at least one set business rule, a first-risk document that meets the conditions is selected from the first sorting sequence.

[0069] In order to effectively obtain first risk documents that are both high-risk and meet specific business needs, as an example, in response to the presence of multiple first documents with the same risk score in the first sorting sequence, based on the degree of matching between the multiple first documents and at least one set business rule, the positions of the multiple first documents in the first sorting sequence are adjusted to obtain a second sorting sequence; according to the selection range under any target category indicated by at least one set business rule, the first risk document that is ranked first and meets the selection range is selected from the second sorting sequence.

[0070] That is to say, when there are multiple first documents with the same risk score in the first sorting sequence, for the first documents with the same risk score, their matching degree with at least one set business rule is evaluated, and the position of the first document in the first sorting sequence is adjusted according to the matching degree between the first document and at least one set business rule. The reimbursement documents with higher matching degree (i.e., documents that are more in line with the business rules) will be arranged in the front, while the documents with lower matching degree will be arranged in the back. Then, according to the set business rules, the selection range of each expense category is determined, and based on the selection range of any target category, the first risk document that is ranked first and meets the selection range is selected from the second sorting sequence corresponding to the any target category. For example, for travel expenses, the business rules are set to stipulate that only the top 10% of reimbursement documents are selected, and the top 10% of reimbursement documents are selected from the second sorting sequence corresponding to travel expenses; for business entertainment expenses, reimbursement documents with an amount exceeding RMB 3,000 are selected, and reimbursement documents with an amount exceeding RMB 3,000 are selected from the second sorting sequence corresponding to business entertainment expenses.

[0071] It should be noted that, in order to ensure that the risk identification and sorting of the documents to be identified are more accurate and reasonable, the matching degree between each first document and the set business rules is evaluated according to the detailed information of each first document, and then the position weight of the document is determined. Finally, the positions of multiple first documents in the first sorting sequence are adjusted according to these position weights to generate a second sorting sequence, as follows:

[0072] 1. For any first document, determine the matching degree between any first document and each set business rule according to the detailed information of the first document;

[0073] In the disclosed embodiment, detailed information of each first invoice is obtained, such as, but not limited to: expense category, expense amount, time information, approval process, etc., and according to the detailed information of the first invoice, the matching between the first invoice and the business rules set by the enterprise is checked item by item. For example, the business rules are set that various expenses exceed the limit, and the transportation expenses and accommodation expenses in the travel expenses both exceed the limit, and the matching degree is relatively high. For another example, the transportation expenses and accommodation expenses in the travel expenses do not exceed the limit, and the matching degree is not high.

[0074] 2. Determine the position weight of any first document according to the matching degree between any first document and each set business rule; wherein the position weight is used to indicate the probability of any first document being arranged in the front;

[0075] In the embodiment of the present disclosure, after determining the degree of match between any first document and each set business rule, the degree of match between any first document and each set business rule can be weighted and summed or averaged to obtain the position weight of any first document, wherein the position weight is used to indicate the probability of any first document being arranged in the front, that is, the higher the position weight, the more likely the first document is to be arranged in the front and given priority; the higher the degree of match, the higher the position weight (because documents with a high degree of match have a higher risk and need to be given priority); the lower the degree of match, the lower the position weight (because documents with a low degree of match have a lower risk and do not need priority review).

[0076] 3. According to the position weight of each first document, the positions of the multiple first documents in the first sorting sequence are adjusted to obtain a second sorting sequence.

[0077] In the embodiment of the present disclosure, the first documents with higher position weights usually have higher risks, and they are placed in front to ensure that these documents are reviewed with priority; the first documents with lower position weights usually have lower risks, and they are placed in the back to reduce unnecessary review work;

[0078] As another example, in response to the fact that there are not multiple first documents with the same risk score in the first sorting sequence, according to the selection range under any target category indicated by at least one set business rule, a first risk document that is ranked first in the first sorting sequence and meets the selection range is selected from the first sorting sequence.

[0079] That is to say, in order to effectively obtain first risk documents that are both high-risk and meet specific business needs, so as to improve financial management efficiency, when there are no multiple first documents with the same risk score in the first sorting sequence, according to the selection range under any expense category indicated by at least one set business rule, and then, according to the selection range, the first risk document that is ranked first and meets the selection range is selected from any target category; for example, at least one set business rule indicates that the top 10% of documents are selected from travel expenses, and the top 10% of documents are selected from the first sorting sequence corresponding to travel expenses; for another example, at least one set business rule indicates that business entertainment expense documents with an amount exceeding 3,000 yuan are selected, and business entertainment expense documents with an amount exceeding 3,000 yuan are screened out from the first sorting sequence corresponding to business entertainment expenses.

[0080] Step 206: Identify risk data from the detailed information of the first risk document and send the risk data to the client.

[0081] It should be noted that the execution process of steps 201 to 202 and step 206 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0082] The risk data identification method of the embodiment of the present disclosure classifies each document to be identified according to the target category of each document to be identified to obtain documents to be identified under at least one target category; for the documents to be identified under any target category, sorts the documents to be identified under any target category according to the risk score of the document to be identified under any target category to obtain a first sorting sequence; determines the first risk document from the first sorting sequence according to at least one set business rule, thereby, through classification and screening, the high-risk first risk documents that need to be handled first can be quickly identified, ensuring that these high-risk documents can be reviewed and processed in time, thereby improving the efficiency of risk data identification.

[0083] In order to clearly illustrate that risk data is identified from detailed information of a first risk document in any embodiment of the present disclosure, the present disclosure proposes a method for identifying risk data.

[0084] Figure 3 This is a flow chart of the risk data identification method provided in the third embodiment of the present disclosure.

[0085] like Figure 3 As shown, the identification of the risk data may include the following steps:

[0086] Step 301: Acquire multiple documents to be identified and detailed information associated with each document to be identified.

[0087] Step 302: for any document to be identified, determine the risk score of any document to be identified according to the detailed information of any document to be identified.

[0088] Step 303: Identify a first risk document from multiple documents to be identified based on at least one set business rule sent by the client and the risk scores of each document to be identified.

[0089] Step 304: In response to the number of the first risk documents being multiple, a second risk document is determined from the first risk documents according to the dialogue interaction content sent by the client.

[0090] In order to improve the level of refinement and personalization of document processing, as an example, when there are multiple first-risk documents, more urgent or important second-risk documents are further screened out based on the dialogue interaction content sent by the client.

[0091] For example, a client (e.g., financial personnel or other relevant personnel) sends interactive content about a first risk document through some interactive method (e.g., chat interface, email, telephone, etc.), where the interactive content may include a detailed explanation of the document, an explanation of the degree of urgency, etc. Then, based on the interactive content, the second risk document is screened out from the first risk document.

[0092] As another example, when there are multiple first risk data, corresponding risk data may be identified from the multiple first risk data respectively.

[0093] Step 305: Identify risk data from the detailed information of the second risk document.

[0094] In the disclosed embodiment, the detailed information of the second risk document is parsed, and risk data directly related to the potential risk is extracted from the detailed information. The risk data may include abnormally high reimbursement amounts, frequent or unreasonable document processing behaviors, document categories or details that do not comply with regulations, document requests that have not been properly approved, etc.

[0095] In order to improve the accuracy of risk data identification, as an example, the intelligent agent can automatically identify the risk data based on the detailed information corresponding to the second risk document, the risk score, and the document verification criteria adapted to the document type of the second risk document.

[0096] In the disclosed embodiment, a prompt template adapted to the business scenario to which the second risk document belongs is obtained; wherein the prompt template is used to prompt the intelligent agent to perform a risk data identification task; according to the detailed information corresponding to the second risk document, the risk score and the document verification criteria adapted to the document type of the second risk document, the prompt template is updated to obtain prompt information; the prompt information is input into the large model for processing to obtain the risk data output by the large model.

[0097] That is to say, a prompt template that matches the business scenario to which the second risk document belongs is obtained, and the prompt template is personalized and updated according to the detailed information of the second risk document (such as the document amount, document category, identity corresponding to the document, etc.), the risk score, and the document verification criteria set by the enterprise (such as reimbursement policy, compliance requirements, etc.), to obtain prompt information. This update process ensures that the prompt template can accurately reflect the situation and risk characteristics of the second risk document, thereby guiding the intelligent agent to perform more accurate risk data identification. Furthermore, the prompt information is input into a large model through the intelligent agent, and the large model performs in-depth analysis and reasoning on the input prompt information, and finally outputs the risk data.

[0098] It should be noted that the intelligent agent includes three key parts: planning, memory and tool use. The planning part is used to decompose the risk data identification task into multiple subtasks when the risk data identification task includes a large number of documents to be identified, and to plan the process of executing the task, and to think and reflect on the process of task execution, so as to decide whether to continue to execute the task, or to judge that the task is completed and terminate the operation; the memory part includes short-term memory and long-term memory. Short-term memory refers to the context in the process of executing the risk data identification task, which will be generated and temporarily stored in the execution process of the subtask and cleared after the task is completed. Long-term memory is information retained for a long time, such as an external knowledge base associated with the risk data identification task; the tool use part is used to equip the intelligent agent with a tool application programming interface (Application Programming Interface, referred to as API), which is used for calculation, query and execution in the execution process of the risk data identification task, such as: calculator, search tool, code executor, database query tool, etc. Based on the above tool API, the intelligent agent can interact with the physical world to perform the risk data identification task.

[0099] It should be noted that the execution process of steps 301 to 303 can be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0100] The method for identifying risk data in the embodiment of the present disclosure, in response to the number of multiple first risk documents, determines the second risk document from the first risk documents according to the dialogue interaction content sent by the client, thereby accurately identifying and prioritizing truly high-risk documents, reducing false alarms and unnecessary review work, and performing personalized review based on the user's interaction content, thereby improving the user experience.

[0101] In order to clearly explain how to determine the risk score of any document to be identified based on the detailed information of any document to be identified in any embodiment of the present disclosure, the present disclosure proposes a method for identifying risk data.

[0102] Figure 4 This is a flow chart of the risk data identification method provided in the fourth embodiment of the present disclosure.

[0103] like Figure 4 As shown, the risk data identification method may include the following steps:

[0104] Step 401, obtaining a plurality of documents to be identified and detailed information associated with each document to be identified.

[0105] Step 402, obtaining reference documents.

[0106] The reference document is of the same document type as any document to be identified, and the reference document does not meet the document verification criteria adapted to the document type to which any document belongs.

[0107] In the disclosed embodiment, the reference document may be a historical violation document of the same document type as any document to be identified, for example, the reference document and any document to be identified are both traffic reimbursement documents; or, the reference document is a violation document of the same document type as any document to be identified and within the same time period.

[0108] Step 403: Determine the similarity between any document to be identified and the reference document based on the first document feature of any document to be identified and the second document feature of the reference document.

[0109] In the disclosed embodiment, the first document feature of any document to be identified refers to the key attributes or information in the detailed information of any document to be identified, such as the reimbursement type (such as travel expenses, meal expenses, office supplies expenses, etc.), amount, date, document object information, document item description, invoice or receipt information, etc. The second document feature refers to the key attributes or information in the detailed information of the reference document, such as an excessively high reimbursement amount, an unreasonable document item description, or invalid invoice information, etc.

[0110] Then, feature matching is performed between the first document feature of any document to be identified and the second document feature of the reference document, and based on the result of feature matching, the similarity between the document to be identified and the reference document is calculated, for example, the cosine similarity, Euclidean distance or Manhattan distance between two feature vectors is calculated.

[0111] Step 404: Determine the risk score of any document to be identified based on the similarity.

[0112] In the embodiment of the present disclosure, the risk score of any document to be identified is evaluated based on the similarity between any document to be identified and a reference document, wherein the higher the similarity, the higher the risk score.

[0113] In order to effectively determine the risk score of any document to be identified, as an example, a risk assessment is performed on any document to be identified based on similarity to obtain the risk level of any document to be identified; and the risk score of any document to be identified is determined based on the risk level of any document to be identified.

[0114] That is to say, based on the calculated similarity, the risk level of any document to be identified is determined. For example, if the similarity is lower than the first set threshold, it is low risk; if the similarity is greater than the first set threshold and less than the second set threshold, it is medium risk; if the similarity is greater than the second set threshold, it is high risk. Furthermore, based on the mapping relationship between the risk level and the risk score, the risk score of any document to be identified is determined.

[0115] Step 405 , identifying a first risk document from multiple documents to be identified based on at least one set business rule sent by the client and the risk scores of each document to be identified.

[0116] Step 406: Identify risk data from the detailed information of the first risk document and send the risk data to the client.

[0117] It should be noted that the execution process of step 401 and steps 405 to 406 can be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0118] The method for identifying risk data of the disclosed embodiment obtains reference documents, determines the similarity between any document to be identified and the reference document based on the first document feature of any document to be identified and the second document feature of the reference document, and determines the risk score of any document to be identified based on the similarity. Thus, the risk score of any document to be identified can be effectively determined, and based on the risk score of any document to be identified, potential non-compliant document processing behaviors can be effectively identified, thereby improving the accuracy and efficiency of document review.

[0119] Based on any embodiment of the present disclosure, taking the document to be identified as a reimbursement document (reimbursement form) as an example, Figure 5 As shown, Figure 5 This is a schematic diagram of the system architecture for implementing the risk data identification method of the embodiment of the present disclosure, and the system mainly includes the following modules:

[0120] 1. Data reading module

[0121] The data reading module is mainly composed of three parts. First, it collects all the invoices and related information, that is, collects the detailed information of the reimbursement documents, such as the relevant information of the reimbursement person, the reimbursement process record information, the audit record information, the review record information, etc. Then, if this information is already in the system, it can be directly pulled from the system and put into the specified table of the database. If it is an offline reimbursement and the invoice is in paper form, it needs to pass optical character recognition (OCR for short); finally, it is entered into the database;

[0122] 2. Collaborative filtering algorithm module

[0123] The collaborative filtering algorithm module mainly consists of two parts. First, it pulls data from the database and performs corresponding data processing to obtain effective features, obtains reference features of reference documents, performs feature correlation analysis on reimbursement documents and reference documents, and finally obtains the risk score of each document.

[0124] 3. Accurate recommendation module

[0125] The precise recommendation module is mainly composed of two parts. First, in order to ensure the accuracy of the recommendation, the system has established a rule engine to support financial auditors to input the required business rules from the front-end interface, such as the minimum number of documents to be drawn from each expense category, the minimum proportion of key expense categories, and other rules; then, based on the set business rules, the rerank algorithm adopts the idea of ​​divide-and-conquer algorithm, and adopts different reranking strategies for different business rules, and finally obtains documents with potential high violation risks;

[0126] For example, if there are reimbursement documents with the same risk score in the sorting of different expense categories, the reimbursement documents with the same risk score are re-sorted based on the degree of match with the business rules. For example, the position of the reimbursement documents with a high degree of match is adjusted forward in the sorting sequence.

[0127] 4. Large model enhanced retrieval module

[0128] The large model enhanced retrieval module is mainly composed of two parts. First, the financial auditor interacts with the financial assistant intelligent agent at the front end to obtain the documents with potential high violation risks through the dialogue. Then, the auditor retrieves the relevant detailed information of the document through the large model for reference, so as to conduct inspection and review more efficiently.

[0129] That is to say, the prompt template is personalized based on the detailed information of the document (such as reimbursement amount, reimbursement category, identity of the person making the reimbursement, etc.), risk score, and expense reimbursement criteria set by the enterprise (such as reimbursement policy, compliance requirements, etc.) to obtain prompt information. Then, the prompt information is input into a large model, which deeply analyzes and infers the input prompt information and finally outputs risk data.

[0130] With the above Figures 1 to 5 Corresponding to the risk data identification method provided in the embodiment, the present disclosure also provides a risk data identification device. Figures 1 to 5 The risk data identification method provided in the embodiment corresponds to the risk data identification method, so the implementation of the risk data identification method is also applicable to the risk data identification device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0131] Figure 6 This is a schematic diagram of the structure of the risk data identification device provided in the fifth embodiment of the present disclosure.

[0132] like Figure 6As shown, the risk data identification device 600 may include: an acquisition module 610 , a determination module 620 , a first identification module 630 and a second identification module 640 .

[0133] Among them, the acquisition module 610 is used to obtain multiple documents to be identified and detailed information associated with each document to be identified; the determination module 620 is used to determine the risk score of any document to be identified based on the detailed information of any document to be identified; the first identification module 630 is used to identify the first risk document from multiple documents to be identified based on at least one set business rule sent by the client and the risk score of each document to be identified; the second identification module 640 is used to identify risk data from the detailed information of the first risk document and send the risk data to the client.

[0134] In a possible implementation of the embodiment of the present disclosure, the first identification module 630 is specifically used to: classify each document to be identified according to a target category of each document to be identified to obtain documents under at least one target category; wherein the target category is determined based on the core fields in the corresponding document to be identified; sort the documents to be identified under any target category according to the risk scores of the documents to be identified under any target category to obtain a first sorting sequence; and determine a first risk document from the first sorting sequence according to at least one set business rule.

[0135] In a possible implementation of the embodiment of the present disclosure, the first identification module 630 is specifically used to: in response to the presence of multiple first documents with the same risk score in the first sorting sequence, adjust the positions of the multiple first documents in the first sorting sequence based on the degree of matching between the multiple first documents and at least one set business rule to obtain a second sorting sequence; and select, from the second sorting sequence, a first risk document that is ranked first and meets the selection range according to a selection range under any target category indicated by at least one set business rule.

[0136] In a possible implementation of the embodiment of the present disclosure, the first identification module 630 is specifically used to: determine the degree of matching between any first document and each set business rule based on the detailed information of the first document; determine the position weight of any first document based on the degree of matching between any first document and each set business rule; wherein the position weight is used to indicate the probability of any first document being arranged in front; and adjust the positions of multiple first documents in the first sorting sequence based on the position weights of each first document to obtain a second sorting sequence.

[0137] In a possible implementation of the embodiment of the present disclosure, the first identification module 630 is specifically used to: in response to the absence of multiple first documents with the same risk score in the first sorting sequence, select the first risk document that is ranked first and meets the selection range from the first sorting sequence according to the selection range under any target category indicated by at least one set business rule.

[0138] In a possible implementation of the embodiment of the present disclosure, the second identification module 640 is specifically used to: in response to the number of first risk documents being multiple, determine the second risk document from the first risk document according to the dialogue interaction content sent by the client; and identify risk data from the detailed information of the second risk document.

[0139] In a possible implementation of the embodiment of the present disclosure, the second identification module 640 is specifically used to: obtain a prompt template adapted to the business scenario to which the second risk document belongs; wherein the prompt template is used to prompt the intelligent agent to perform a risk data identification task; update the prompt template according to the detailed information corresponding to the second risk document, the risk score, and the document verification criteria adapted to the document type of the second risk document to obtain prompt information; and input the prompt information into the large model for processing to obtain the risk data output by the large model.

[0140] In a possible implementation of the embodiment of the present disclosure, the determination module 620 is specifically used to: obtain a reference document; wherein the reference document is of the same document type as any document to be identified, and the reference document does not comply with the document verification criteria adapted to the document type to which any document belongs; determine the similarity between any document to be identified and the reference document based on a first document feature of any document to be identified and a second document feature of the reference document; and determine a risk score for any document to be identified based on the similarity.

[0141] In a possible implementation of the embodiment of the present disclosure, the determination module 620 is specifically used to: perform risk assessment on any document to be identified based on similarity to obtain the risk level of any document to be identified; and determine the risk score of any document to be identified based on the risk level of any document to be identified.

[0142] In a possible implementation of the embodiment of the present disclosure, the acquisition module 610 is specifically used to: receive multiple documents to be identified sent by the client; extract information of any document to be identified to obtain document information of any document to be identified; obtain associated information associated with any document to be identified; and use the document information and associated information of any document to be identified as detailed information of any document to be identified.

[0143] In a possible implementation manner of the embodiment of the present disclosure, the associated information includes at least one of the following information: document object information; document processing process record information; document audit record information; document review record information.

[0144] The risk data identification device of the disclosed embodiment calculates a risk score according to the detailed information of the document to be identified, and then identifies a first risk document with high risk that meets user needs from multiple documents to be identified based on set business rules and risk scores. Finally, the risk data is located from the detailed information of the first risk document, and the risk data is sent to the client, so that relevant personnel can have a clearer understanding of the risk situation of the document to be identified, thereby improving the interpretability of risk data identification, reducing the erroneous processing of documents, improving financial management efficiency, and enhancing the financial control and risk management capabilities of the enterprise, while meeting user needs and improving user experience.

[0145] In order to implement the above embodiments, the present disclosure also provides an electronic device, which may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the risk data identification method proposed in any of the above embodiments of the present disclosure.

[0146] In order to implement the above embodiments, the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the risk data identification method proposed in any of the above embodiments of the present disclosure.

[0147] In order to implement the above embodiments, the present disclosure further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the risk data identification method proposed in any of the above embodiments of the present disclosure.

[0148] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0149] Figure 7A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0150] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 702 or a computer program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An I / O (Input / Output) interface 705 is also connected to the bus 704.

[0151] Multiple components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0152] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the above-mentioned risk data identification method. For example, in some embodiments, the above-mentioned risk data identification method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the risk data identification method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the above-mentioned risk data identification method in any other appropriate manner (for example, by means of firmware).

[0153] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0154] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0155] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0156] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0157] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0158] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0159] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0160] According to the technical solution of the embodiment of the present disclosure, a risk score is calculated based on the detailed information of the document to be identified, and then a first risk document with high risk that meets user needs is identified from multiple documents to be identified based on the set business rules and risk scores. Finally, risk data is located from the detailed information of the first risk document, and the risk data is sent to the client, so that relevant personnel can have a clearer understanding of the risk situation of the document to be identified, which improves the interpretability of risk data identification, reduces document error processing, improves financial management efficiency, enhances the company's financial control and risk management capabilities, and at the same time meets user needs and improves user experience.

[0161] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions proposed in this disclosure can be achieved, and this document does not limit this.

[0162] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for identifying risk data, comprising: Acquire multiple documents to be identified and detailed information associated with each of the documents to be identified; For any document to be identified, determine the risk score of the document to be identified based on the detailed information of the document to be identified; Identifying a first risk document from the multiple documents to be identified according to at least one set business rule sent by the client and the risk score of each of the documents to be identified; Risk data is identified from the detailed information of the first risk document, and the risk data is sent to the client.

2. The method according to claim 1, wherein: The step of identifying a first risk document from the multiple documents to be identified according to at least one set business rule sent by the client and the risk score of each of the documents to be identified includes: Classifying each of the documents to be identified according to the target category of each document to be identified, so as to obtain documents to be identified under at least one target category; wherein the target category is determined according to the core fields in the corresponding document to be identified; sorting the documents to be identified under any target category according to the risk scores of the documents to be identified under the target category to obtain a first sorting sequence; A first risk document is determined from the first sorting sequence according to the at least one set business rule.

3. The method according to claim 2, wherein: The step of determining a first risk document from the first sorting sequence according to the at least one set business rule comprises: In response to the presence of a plurality of first documents with the same risk score in the first sorting sequence, based on the matching degree between the plurality of first documents and the at least one set business rule, positions of the plurality of first documents in the first sorting sequence are adjusted to obtain a second sorting sequence; According to the selection range under any target category indicated by the at least one set business rule, a first risk document that is ranked first and meets the selection range is selected from the second sorting sequence.

4. The method according to claim 3, wherein: In response to the presence of a plurality of first documents with the same risk score in the first sorting sequence, based on the matching degree between the plurality of first documents and the at least one set business rule, positions of the plurality of first documents in the first sorting sequence are adjusted to obtain a second sorting sequence, including: Determining, according to the detailed information of any first document, the degree of matching between the any first document and each set business rule; Determine the position weight of any first document according to the matching degree between any first document and each set business rule; wherein the position weight is used to indicate the probability that any first document is arranged in the front; According to the position weight of each of the first documents, the positions of the plurality of first documents in the first sorting sequence are adjusted to obtain a second sorting sequence.

5. The method according to claim 2, wherein: The step of determining a first risk document from the first sorting sequence according to the at least one set business rule comprises: In response to the fact that there are no multiple first documents with the same risk score in the first sorting sequence, according to the selection range under any target category indicated by the at least one set business rule, a first risk document that is ranked first in the first sorting sequence and meets the selection range is selected from the first sorting sequence.

6. The method according to claim 1, wherein: The step of identifying risk data from the detailed information of the first risk document includes: In response to the number of the first risk documents being multiple, determining a second risk document from the first risk documents according to the dialogue interaction content sent by the client; Risk data is identified from the detailed information of the second risk document.

7. The method according to claim 6, wherein: The step of identifying risk data from the detailed information of the second risk document includes: Acquire a prompt template adapted to the business scenario to which the second risk document belongs; wherein the prompt template is used to prompt the agent to perform a risk data identification task; According to the detailed information corresponding to the second risk document, the risk score, and the document verification criteria adapted to the document type of the second risk document, the prompt template is updated to obtain prompt information; The prompt information is input into the large model for processing to obtain the risk data output by the large model.

8. The method according to claim 1, wherein: The step of determining, for any document to be identified, a risk score of the document to be identified based on detailed information of the document to be identified, includes: Acquire a reference document; wherein the reference document is of the same document type as any document to be identified, and the reference document does not meet the document verification criteria adapted to the document type to which any document belongs; Determining the similarity between the any document to be identified and the reference document according to the first document feature of the any document to be identified and the second document feature of the reference document; A risk score of any document to be identified is determined based on the similarity.

9. The method according to claim 8, wherein: Determining the risk score of any document to be identified according to the similarity includes: According to the similarity, risk assessment is performed on any document to be identified to obtain a risk level of any document to be identified; A risk score of any document to be identified is determined according to the risk level of any document to be identified.

10. The method according to claim 1, wherein: The obtaining of a plurality of documents to be identified and detailed information associated with each of the documents to be identified includes: Receiving the multiple documents to be identified sent by the client; For any document to be identified, extract information of the document to be identified to obtain document information of the document to be identified; Acquire association information associated with any of the documents to be identified; The document information and associated information of any document to be identified are used as detailed information of any document to be identified.

11. The method according to claim 10, wherein: The association information includes at least one of the following information: Document object information; Record information during document processing; Document review record information; Document review record information.

12. A risk data identification device, comprising: An acquisition module, used for acquiring a plurality of documents to be identified and detailed information associated with each of the documents to be identified; A determination module, configured to determine, for any document to be identified, a risk score of the document to be identified based on detailed information of the document to be identified; A first identification module, configured to identify a first risk document from the multiple documents to be identified according to at least one set business rule sent by the client and the risk score of each of the documents to be identified; The second identification module is used to identify risk data from the detailed information of the first risk document and send the risk data to the client.

13. An electronic device, wherein: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images