Financial electronic bill abnormal information identification method, system and device and medium
Through data augmentation and fine-tuning training of Qwen-7b large language model, combined with RAG technology, the flexibility and accuracy of the existing technology in dealing with complex and frequent changes in fiscal electronic bills is solved, and the financial electronic bills is realized. Quick and accurate identification and judgment reduce the risk of human error.
Patent Information
- Application Number
- CN202411591474.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art is difficult to flexibly deal with unstructured data when dealing with complex and frequently changing financial electronic bill anomaly information, and it is highly dependent on the accuracy of NLP technology and machine learning models, resulting in limited adaptability in real-time and dynamic changes.
Using data augmentation technology to process bill data, convert it to JSON format, and fine-tuning training is used to use Qwen-7b large language model for fine-tuning training, design fine-tuning training prompt words for key bill information, combine RAG technology for model reasoning, improve the language understanding and generation of the model ability.
It improves the accuracy and efficiency of data processing, realizes the rapid and accurate identification and judgment of fiscal electronic bills, reduces the burden of manual review, and reduces the risk of human error.
Smart Images

Figure CN119091454A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method, system, equipment and medium for identifying abnormal information of financial electronic bills, and belongs to the field of financial electronic bill supervision. Background Art
[0002] With the development of information technology, fiscal electronic invoices have become an important part of modern fiscal management. They not only improve the efficiency of invoice processing, but also reduce the use of paper invoices and the possibility of human errors. However, with the popularization of electronic invoices, the detection and processing of abnormal information has also become a key issue. Abnormal information includes forgery, tampering, repeated submission, expiration, etc. of invoices. If these abnormal information are not discovered in time, they may lead to the loss of fiscal funds and confusion in financial management.
[0003] At present, traditional methods for identifying abnormal information mainly include rule-based methods and statistical methods. Rule-based methods screen bills through preset rules, which are usually based on experience and historical data. However, this method is difficult to deal with complex and changing abnormal patterns and new fraud methods. Statistical methods discover abnormal patterns through data analysis, but these methods usually require a large amount of historical data and strong computing power, and have limited adaptability to real-time and dynamic changes.
[0004] Prior art, such as the US patent number "US10795752B2", discloses a method for document verification. The method includes the following steps: extracting journal entry data and supporting documents to be verified, including reconciliation to identify potential errors. Using natural language processing technology to extract entities related to the ledger, determine the values of corresponding data fields and their mapping relationships. When there is a one-to-one mapping, select the corresponding value; otherwise, obtain enhanced data and process it to determine the value. Compare the value with the actual value and generate a potential error notification. Process the data through rules and machine learning to detect anomalies and identify contextual information of behavioral deviations.
[0005] The problem with the above existing technologies is that they are highly dependent on the accuracy of NLP technology and machine learning models, and the performance of these models may be affected by many factors such as data quality, model design, training data, etc. This approach can be very effective when processing structured or semi-structured data by processing data through detailed classification and mapping logic. However, this approach may become inflexible when faced with highly unstructured or frequently changing data. Summary of the invention
[0006] In order to solve the above problems existing in the prior art, the present invention proposes a method, system, device and medium for identifying abnormal information of fiscal electronic invoices.
[0007] The technical solution of the present invention is as follows: In one aspect, the present invention provides a method for identifying abnormal information of financial electronic invoices, comprising the following steps: Collect bill data, filter out bill data with invalid or erroneous formats, and extract key features of the filtered bill data; bill data includes fiscal policy regulations, laws and regulations, policy interpretations, and historical bill data, and key features include billing units, bill types, total bill amounts, bill items, item amounts, remarks, and secondary item information; Use data enhancement technology on the bill data to obtain enhanced bill data, and convert the enhanced bill data into a dataset in JSON format; Use the dataset to fine-tune the Qwen-7b language model and design fine-tune training prompts for key information in the bill to guide the Qwen-7b language model to better focus on and extract this information; The fiscal electronic invoices are judged based on the trained Qwen-7b large language model and the recognition results are output.
[0008] As a preferred embodiment, the data enhancement techniques include back translation, synonym replacement and Cartesian set; Back translation: Use translation tools to translate the notes or secondary project information in the bill data into other languages multiple times, and then translate the other languages back into Chinese to obtain a similar expression of the sentence; Synonym replacement: replace the keywords in the notes in the bill data or the secondary project information with synonyms to form new samples; the keywords include but are not limited to donation, sponsorship and fundraising; Cartesian set: The suspicious data in the tampered bill data and the random bill data are combined into a Cartesian set to form a new suspicious sample.
[0009] As a preferred implementation, the fine-tuning training method of the Qwen-7b large language model is: Divide the dataset in JSON format into a training dataset and a validation dataset; Use the Transformers library to load the Qwen-7b model; Configure the learning rate, batch size, and training rounds for training; Load the pre-trained QLora model; Write a training loop to fine-tune the Qwen-7b model using the QLora model and training dataset, and record the weights during the fine-tuning process; Use the validation set to evaluate the performance of the fine-tuned model and adjust the weights during fine-tuning based on the model performance.
[0010] As a preferred implementation, RAG technology is combined in the model reasoning process, specifically, information in a related knowledge base or database is retrieved through RAG technology to enhance the generation capability of the model.
[0011] As a preferred implementation, the prompt word includes the project name and remarks content; The prompt words are saved in the database and injected into the Qwen-7b model through the prepared statements of the MyBatis framework.
[0012] On the other hand, the present invention also provides a fiscal electronic bill abnormal information identification system, comprising: The data collection module collects bill data, filters out bill data with invalid or erroneous formats, and extracts key features of the filtered bill data; the bill data includes fiscal policy regulations, laws and regulations, policy interpretations, and historical bill data, and key features include billing units, bill types, total bill amounts, bill items, item amounts, remarks, and secondary item information; The data processing module uses data enhancement technology to obtain enhanced bill data, and converts the enhanced bill data into a data set in JSON format; The model training module uses the dataset to fine-tune the Qwen-7b large language model and designs fine-tuning training prompts for key information in the bill to guide the Qwen-7b large language model to better focus on and extract this information; The result output module judges the fiscal electronic invoice based on the trained Qwen-7b large language model and outputs the recognition results.
[0013] On the other hand, the present invention further provides an electronic device having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for identifying abnormal information of fiscal electronic invoices as described in any embodiment of the present invention.
[0014] On the other hand, the present invention also provides a computer-readable medium for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the method for identifying abnormal information of fiscal electronic invoices as described in any embodiment of the present invention.
[0015] The present invention has the following beneficial effects: The present invention uses data enhancement technology to process bill data, which helps to increase the diversity and richness of the data set, thereby improving the accuracy and robustness of subsequent model training. Through data enhancement, the model can learn more diverse data features, so that it performs better when processing actual data. Fine-tuning training is performed using the large language model Qwen-7b, which has powerful language understanding and generation capabilities. By designing fine-tuning training prompts for key information in the bill, the model can be guided to better focus on and extract this information, further improving the accuracy and efficiency of data processing.
[0016] The automatic processing and judgment of bill data is realized, which greatly reduces the burden of manual review. Through the trained Qwen-7b large language model, fiscal electronic bills can be quickly and accurately identified and judged, and the recognition results can be output. This not only improves work efficiency, but also reduces the risk of human error. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0019] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.
[0020] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0021] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0022] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.
[0023] Embodiment 1: See also Figure 1 The present invention provides a method for identifying abnormal information of financial electronic bills, comprising the following steps: Collect bill data, filter out bill data with invalid or erroneous formats, and extract key features of the filtered bill data; bill data includes fiscal policy regulations, laws and regulations, policy interpretations, and historical bill data, and key features include billing units, bill types, total bill amounts, bill items, item amounts, remarks, and secondary item information; Use data enhancement technology on the bill data to obtain enhanced bill data, and convert the enhanced bill data into a dataset in JSON format; Use the dataset to fine-tune the Qwen-7b language model and design fine-tune training prompts for key information in the bill to guide the Qwen-7b language model to better focus on and extract this information; The fiscal electronic invoices are judged based on the trained Qwen-7b large language model and the recognition results are output.
[0024] As a preferred embodiment, the data enhancement techniques include back translation, synonym replacement and Cartesian set; Back translation: Use translation tools to translate the notes or secondary project information in the bill data into other languages multiple times, and then translate the other languages back into Chinese to obtain a similar expression of the sentence; Synonym replacement: replace the keywords in the notes in the bill data or the secondary project information with synonyms to form new samples; the keywords include but are not limited to donation, sponsorship and fundraising; Cartesian set: The suspicious data in the tampered bill data and the random bill data are combined into a Cartesian set to form a new suspicious sample.
[0025] Data augmentation, also known as data expansion, is a technique to artificially increase the training set by creating a modified copy of the data set using existing data. It is one of the commonly used techniques in deep learning, including making minor changes to the data set or using deep learning to generate new data points. Data augmentation is mainly used to increase the training data set, making the data set as diverse as possible, so that the trained model has stronger generalization ability.
[0026] Because the suspicious data set is small, we mainly perform data enhancement on suspicious bills to achieve a basic balance between suspicious bills and normal bills.
[0027] As a preferred implementation, the fine-tuning training method of the Qwen-7b large language model is: Divide the dataset in JSON format into a training dataset and a validation dataset; Use the Transformers library to load the Qwen-7b model; Configure the learning rate, batch size, and training rounds for training; Load the pre-trained QLora model; Write a training loop to fine-tune the Qwen-7b model using the QLora model and training dataset, and record the weights during the fine-tuning process; Use the validation set to evaluate the performance of the fine-tuned model and adjust the weights during fine-tuning based on the model performance.
[0028] Qwen-7b features: This model is based on the Transformer structure, has 7 billion parameters, and performs well on multiple natural language processing tasks. Its powerful text generation and understanding capabilities provide a solid foundation for the identification of suspicious fiscal bills.
[0029] As a preferred implementation, RAG technology is combined in the model reasoning process, specifically, information in a related knowledge base or database is retrieved through RAG technology to enhance the generation capability of the model.
[0030] As a preferred implementation, the prompt word includes the project name and remarks content; The prompt words are saved in the database and injected into the Qwen-7b model through the prepared statements of the MyBatis framework.
[0031] This approach helps prevent SQL injection attacks because it sets parameter values through prepared statements instead of directly splicing parameter values into the SQL statement.
[0032] Embodiment 2: The present invention also provides a fiscal electronic bill abnormal information identification system, comprising: The data collection module collects bill data, filters out bill data with invalid or erroneous formats, and extracts key features of the filtered bill data; the bill data includes fiscal policy regulations, laws and regulations, policy interpretations, and historical bill data, and key features include billing units, bill types, total bill amounts, bill items, item amounts, remarks, and secondary item information; The data processing module uses data enhancement technology to obtain enhanced bill data, and converts the enhanced bill data into a data set in JSON format; The model training module uses the dataset to fine-tune the Qwen-7b large language model and designs fine-tuning training prompts for key information in the bill to guide the Qwen-7b large language model to better focus on and extract this information; The result output module judges the fiscal electronic invoice based on the trained Qwen-7b large language model and outputs the recognition results.
[0033] Embodiment three: This embodiment provides an electronic device having a computer program stored thereon, and when the computer program is executed by a processor, the method for identifying abnormal information of fiscal electronic invoices as described in any embodiment of the present invention is implemented.
[0034] Embodiment 4: This embodiment provides a computer-readable medium for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method for identifying abnormal information of fiscal electronic invoices as described in any embodiment of the present invention.
[0035] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.
[0036] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0037] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0038] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.
[0039] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for identifying abnormal information of financial electronic invoices, characterized in that: The following steps are involved: Collect bill data, filter out bill data with invalid or erroneous formats, and extract key features of the filtered bill data; among them, bill data includes fiscal policy regulations, laws and regulations, policy interpretations, and historical bill data, and key features include billing units, bill types, total bill amounts, bill items, item amounts, remarks information, and secondary item information; use data enhancement technology to obtain enhanced bill data, and convert the enhanced bill data into a JSON formatted dataset; use the dataset to fine-tune the Qwen-7b large language model, and design fine-tuning training prompts for key information in the bill to guide the Qwen-7b large language model to better focus on and extract this information; judge the fiscal electronic bills based on the trained Qwen-7b large language model, and output the recognition results.
2. The method for identifying abnormal information of financial electronic invoices according to claim 1 is characterized in that: The data enhancement techniques include back translation, synonym substitution, and Cartesian sets; Back translation: Use translation tools to translate the notes or secondary project information in the bill data into other languages multiple times, and then translate the other languages back into Chinese to obtain a similar expression of the sentence; Synonym replacement: replace the keywords in the notes in the bill data or the secondary project information with synonyms to form new samples; the keywords include but are not limited to donation, sponsorship and fundraising; Cartesian set: The suspicious data in the tampered bill data and the random bill data are combined into a Cartesian set to form a new suspicious sample.
3. The method for identifying abnormal information of financial electronic invoices according to claim 1 is characterized in that: The fine-tuning training method of the Qwen-7b large language model is: Divide the dataset in JSON format into a training dataset and a validation dataset; Use the Transformers library to load the Qwen-7b model; Configure the learning rate, batch size, and training rounds for training; Load the pre-trained QLora model; Write a training loop to fine-tune the Qwen-7b model using the QLora model and training dataset, and record the weights during the fine-tuning process; Use the validation set to evaluate the performance of the fine-tuned model and adjust the weights during fine-tuning based on the model performance.
4. The method for identifying abnormal information of financial electronic invoices according to claim 3 is characterized in that: The RAG technology is combined in the model reasoning process, specifically, the information in the relevant knowledge base or database is retrieved through the RAG technology to enhance the generation capability of the model.
5. The method for identifying abnormal information of financial electronic invoices according to claim 1 is characterized in that: The prompt words include the project name and remarks; The prompt words are saved in the database and injected into the Qwen-7b model through the prepared statements of the MyBatis framework.
6. A fiscal electronic bill abnormal information identification system, characterized in that: include: The data collection module collects bill data, filters out bill data with invalid or erroneous formats, and extracts key features of the filtered bill data; the bill data includes fiscal policy regulations, laws and regulations, policy interpretations, and historical bill data, and key features include billing units, bill types, total bill amounts, bill items, item amounts, remarks, and secondary item information; The data processing module uses data enhancement technology to obtain enhanced bill data, and converts the enhanced bill data into a data set in JSON format; The model training module uses the dataset to fine-tune the Qwen-7b large language model and designs fine-tuning training prompts for key information in the bill to guide the Qwen-7b large language model to better focus on and extract this information; The result output module judges the fiscal electronic invoice based on the trained Qwen-7b large language model and outputs the recognition results.
7. The financial electronic invoice abnormal information identification system according to claim 6 is characterized in that: The model training module, the fine-tuning training method of the Qwen-7b large language model is: Divide the dataset in JSON format into a training dataset and a validation dataset; Use the Transformers library to load the Qwen-7b model; Configure the learning rate, batch size, and training rounds for training; Load the pre-trained QLora model; Write a training loop to fine-tune the Qwen-7b model using the QLora model and training dataset, and record the weights during the fine-tuning process; Use the validation set to evaluate the performance of the fine-tuned model and adjust the weights during fine-tuning based on the model performance.
8. The financial electronic invoice abnormal information identification system according to claim 7 is characterized in that: The model training module combines RAG technology in the model reasoning process, specifically, retrieves information in the relevant knowledge base or database through RAG technology to enhance the generation capability of the model.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for identifying abnormal information of fiscal electronic invoices as described in any one of claims 1 to 5 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for identifying abnormal information of fiscal electronic invoices as described in any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Suspicious electronic bill prediction method based on multi-model fusion
CN116739602A
Bill identification method and device, computer equipment and storage medium
CN118155199A
Uniform Data Model and API for Representation and Processing of Semantic Data
US20140040295A1
