Bill recognition model training method and bill analysis method

Abstract

The present application is applicable to the field of image recognition technology, and provides a bill recognition model training method and a bill analysis method, including: obtaining a bill training set, performing image recognition on the bill image in the bill training set, obtaining a first bill recognition result, and using the first bill recognition result, a preset prompt word, and preset standard merchant information to perform model training on a preset large language model to obtain a bill recognition model; the preset standard merchant information at least includes a complete merchant name, and the preset prompt word is used to guide the preset large language model to extract at least the merchant name from the first bill recognition result, and guide the preset large language model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information. The present application can improve the accuracy of bill recognition.

Description

Technical Field

[0001] The present application belongs to the field of image recognition technology, and in particular relates to a bill recognition model training method, a bill analysis method, a device, an electronic device and a computer program product. Background Art

[0002] With the continuous development of e-commerce, the digital identification and analysis of bills are becoming more and more important. Merchants or platforms usually provide bill identification and analysis services, and users can directly obtain the corresponding bill information based on the uploaded bill image, thereby improving the user's shopping experience.

[0003] Currently, the bill information in the bill image can be extracted through image recognition technology. However, in the image recognition process, since the "merchant name" in the bill image is often a font icon or a special-shaped character, the "merchant name" is not fully recognized, which seriously affects the accuracy of bill recognition. Summary of the invention

[0004] The embodiments of the present application provide a bill recognition model training method, a bill analysis method, a device, an electronic device and a computer program product, which can improve the accuracy of bill recognition.

[0005] In a first aspect, an embodiment of the present application provides a bill recognition model training method, comprising:

[0006] Acquire a bill training set, perform image recognition on bill images in the bill training set, and obtain a first bill recognition result;

[0007] The preset large language model is trained using the first bill recognition result, the preset prompt word and the preset standard merchant information to obtain a bill recognition model; the preset standard merchant information at least includes a complete merchant name, and the preset prompt word is used to guide the preset large language model to extract at least the merchant name from the first bill recognition result, and guide the preset large language model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information.

[0008] In a second aspect, an embodiment of the present application provides a bill analysis method, including:

[0009] Acquire a bill image to be identified, perform image recognition on the bill image to be identified, and obtain a second bill recognition result;

[0010] The second bill recognition result, a preset prompt word and preset standard merchant information are used as inputs of a bill recognition model to obtain a bill recognition result output by the bill recognition model, wherein the preset standard merchant information at least includes a complete merchant name, the bill recognition model extracts at least a merchant name from the second bill recognition result according to the preset prompt word, and matches the extracted merchant name with the complete merchant name included in the preset standard merchant information according to the preset prompt word, and the bill recognition model is obtained according to the bill recognition model training method as described in the first aspect.

[0011] In a third aspect, an embodiment of the present application provides a bill recognition model training device, comprising:

[0012] A training data acquisition module, used to acquire a bill training set, perform image recognition on bill images in the bill training set, and obtain a first bill recognition result;

[0013] A model training module is used to perform model training on a preset large language model using the first bill recognition result, a preset prompt word and preset standard merchant information to obtain a bill recognition model; the preset standard merchant information at least includes a complete merchant name, and the preset prompt word is used to guide the preset large language model to extract at least the merchant name from the first bill recognition result, and guide the preset large language model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information.

[0014] In a fourth aspect, an embodiment of the present application provides a bill analysis device, including:

[0015] An image recognition module is used to obtain a bill image to be recognized, perform image recognition on the bill image to be recognized, and obtain a second bill recognition result;

[0016] A bill recognition module, used to use the second bill recognition result, a preset prompt word and preset standard merchant information as inputs of a bill recognition model to obtain a bill recognition result output by the bill recognition model, wherein the preset standard merchant information at least includes a complete merchant name, the bill recognition model extracts at least a merchant name from the second bill recognition result according to the preset prompt word, and matches the extracted merchant name with the complete merchant name included in the preset standard merchant information according to the preset prompt word, and the bill recognition model is obtained according to the bill recognition model training method as described in the first aspect.

[0017] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect or the second aspect when executing the computer program.

[0018] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0019] In a seventh aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the method described in the first aspect or the second aspect above.

[0020] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0021] The present application trains the preset large language model through the first bill recognition result, the preset prompt word and the preset standard merchant information to obtain the bill recognition model, which can improve the accuracy of bill recognition. Specifically, when performing model training, since the above-mentioned preset standard merchant information at least includes a complete merchant name, and the above-mentioned preset prompt word is used to guide the preset large language model to extract at least the merchant name from the first bill recognition result, and guide the preset large language model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information, it means that through the above-mentioned preset prompt word, the bill recognition model can learn the matching relationship between the merchant name in the first bill recognition result and the complete merchant name in the preset standard merchant information, so that the trained bill recognition model can recognize the complete merchant name, thereby avoiding the problem of incomplete recognition of the merchant name and improving the accuracy of bill recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 It is a flowchart of a bill recognition model training method provided in an embodiment of the present application;

[0024] Figure 2 It is a flow chart of a bill analysis method provided in an embodiment of the present application;

[0025] Figure 3 It is a schematic diagram of the process of automatic bill points provided in the embodiment of the present application;

[0026] Figure 4 It is a structural schematic diagram of a bill recognition model training device provided in an embodiment of the present application;

[0027] Figure 5 is a schematic diagram of the structure of a bill analysis device provided in an embodiment of the present application;

[0028] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0030] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0031] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0032] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0033] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0034] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0035] In the bill recognition and analysis services provided by merchants or platforms, users can directly obtain the corresponding bill information based on the uploaded bill image. At the same time, merchants or platforms can automatically perform bill analysis based on the recognized bill information, thereby improving the user's shopping experience. For example, in the automatic points accumulation scenario for shopping bills, merchants or platforms can recognize bill information such as merchant name, transaction time, transaction amount, and transaction voucher based on the bill image uploaded by the user, and then automatically accumulate points based on this bill information, thereby improving the user's retention rate. Among them, accurate recognition of bill information in bill images is crucial for bill analysis.

[0036] At present, the following methods are mainly used to extract bill information from bill images: 1. Image recognition technology: mainly uses deep neural networks to detect text areas in bill images, further segment text lines within the detected text areas, extract feature sequences of text lines, and convert feature sequences into specific character text. 2. Template matching technology: for shopping bills of different formats, different templates are pre-defined to quickly locate and extract bill information through template matching.

[0037] However, in image recognition technology, each text box needs to be segmented and the location and category of the text box need to be marked, which makes data annotation difficult. In addition, in the image recognition process, some bill information (such as "merchant name") is often a font icon (such as a logo icon) or a special-shaped character in the bill image. At the same time, the bill images uploaded by users come from different sources and are often blurred or wrinkled, which makes some bill information (such as "merchant name") incomplete and inaccurate, thus affecting the accuracy of bill recognition. In template matching technology, templates need to be defined for each format of shopping bills. When the format of the shopping bill is updated, the template needs to be updated synchronously. The adaptability is poor, and inaccurate bill recognition often occurs. Therefore, the above method will lead to inaccurate bill recognition.

[0038] In order to improve the accuracy of bill recognition, the present application provides a bill recognition model training method, in which a bill training set is obtained, image recognition is performed on the bill images in the bill training set to obtain a first bill recognition result, and a preset large language model is trained using the above-mentioned first bill recognition result, preset prompt words and preset standard merchant information to obtain a bill recognition model, wherein the above-mentioned preset standard merchant information at least includes a complete merchant name, and the above-mentioned preset prompt words are used to guide the preset large language model to extract the merchant name from the first bill recognition result, and guide the preset large language model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information.

[0039] Figure 1 A flow chart of a bill recognition model training method provided by an embodiment of the present application is shown, and is described in detail as follows:

[0040] S11. Obtain a bill training set, perform image recognition on bill images in the bill training set, and obtain a first bill recognition result.

[0041] The above-mentioned receipt training set may include different types of receipt images. For example, shopping receipt image samples from different retailers, different sizes, and different printing qualities may be collected as the receipt training set. The above-mentioned shopping receipt image samples may cover receipt information in different languages, fonts, font sizes, and layouts. The receipt information included in the above-mentioned shopping receipt image samples may include at least one of the following: product name, price, quantity, order number, serial number, barcode, date, merchant information, etc. At the same time, the above-mentioned shopping receipt image samples may include photographed pictures of paper shopping receipts and shopping screenshots using program applications, etc.

[0042] Specifically, the first bill recognition result can be obtained by performing image recognition on different types of bill images included in the bill training set through a preset image recognition method. Optionally, the preset image recognition method can include one of optical character recognition technology, template matching technology, and neural network model recognition technology. The first bill recognition result can include bill text recognition results (such as merchant name, product name, transaction amount, quantity, order number, date, etc.), bill pattern recognition results (such as recognized logo pattern, barcode, QR code, etc.), etc.

[0043] In the embodiment of the present application, by performing image recognition on different types of bill images, the first bill recognition result can include bill information of different types of bill images, thereby improving the generalization ability of the bill recognition model during training.

[0044] S12. Performing model training on a preset large language model using the first bill recognition result, preset prompt words, and preset standard merchant information to obtain a bill recognition model; the preset standard merchant information at least includes a complete merchant name, and the preset prompt words are used to guide the preset large language model to extract at least a merchant name from the first bill recognition result, and guide the preset large language model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information.

[0045] Among them, the above-mentioned preset standard merchant information may include complete merchant names, complete merchant addresses and other information. For example, the complete merchant names of all merchants in a supermarket or shopping platform can be collected to obtain a list of standard merchant names, and the above-mentioned standard merchant list is used as the preset standard merchant information. The above-mentioned preset prompt words refer to text or instructions that provide input to the preset large language model to guide it to generate specific outputs. The above-mentioned preset prompt words can be used to constrain or guide the model execution capabilities of the preset large language model, including but not limited to at least one of the following: model execution steps, model extraction range, model output range, model output format, etc.

[0046] Specifically, when training the preset large language model, training data in a preset format can be constructed based on the above-mentioned first bill recognition result, the preset prompt words and the preset standard merchant information, the above-mentioned training data is input into the preset large language model, and the parameters of the training process are set. The above-mentioned preset prompt words can guide the preset large language model to extract the merchant name and other field information from the first bill recognition result during training, and then guide the above-mentioned preset large language model to map and match the extracted merchant name with the complete merchant name in the preset standard merchant information, until the training is stopped when the preset requirements are met, and the trained bill recognition model is obtained. Optionally, the above-mentioned preset format can be in json format.

[0047] The parameters of the training process may include: learning rate, number of training data traversal (epochs), batch size (batch size), model weight (query_key_value), etc. The preset requirements may include loss value requirements or iteration number requirements, etc.

[0048] In some optional embodiments, the above-mentioned preset large language model can be one of the following: ChatGLM4, Llama, Qwen2.5 and GPT, etc.

[0049] The present application trains the preset large language model through the first bill recognition result, the preset prompt word and the preset standard merchant information to obtain the bill recognition model, which can improve the accuracy of bill recognition. Specifically, when performing model training, since the above-mentioned preset standard merchant information at least includes a complete merchant name, and the above-mentioned preset prompt word is used to guide the preset large language model to extract at least the merchant name from the first bill recognition result, and guide the preset large language model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information, it means that through the above-mentioned preset prompt word, the bill recognition model can learn the matching relationship between the merchant name in the first bill recognition result and the complete merchant name in the preset standard merchant information, so that the trained bill recognition model can recognize the complete merchant name, thereby avoiding the problem of incomplete recognition of the merchant name and improving the accuracy of bill recognition.

[0050] In some embodiments, performing image recognition on the bill images in the bill training set to obtain a first bill recognition result includes:

[0051] The preset optical character recognition model is used to perform text recognition on the bill images in the above-mentioned bill training set to obtain the above-mentioned first bill recognition result; the above-mentioned first bill recognition result at least includes a bill text recognition result.

[0052] Specifically, the text information contained in different types of bill images in the bill training set can be recognized by a preset optical character recognition (OCR) model to obtain a first bill recognition result including a bill text recognition result (such as a merchant name, a product name, a transaction amount, a quantity, an order number, a date, etc.). In addition, the first bill recognition result may also include a bill position recognition result (such as the position information of the recognized text or pattern) and a confidence level corresponding to the bill text recognition result, etc., wherein the confidence level is used to reflect the accuracy of the bill text recognition result.

[0053] For example, assuming that the bill image A is a photo of a paper receipt printed by a POS machine, the OCR model can output the first bill recognition result of the bill image A, including: [[[299.0,1005.0],[624.0,1015.0],[622.0,1065.0],[297.0,1055.0]],('Amount payable: 38',0.9)], wherein 'Amount payable: 38' is the recognized bill text recognition result, 0.9 is the confidence level corresponding to the bill text recognition result, and [299.0,1005.0],[624.0,1015.0],[622.0,1065.0],[297.0,1055.0] is the location information corresponding to the bill text recognition result.

[0054] In the embodiment of the present application, the text information in the bill image can be quickly recognized through a preset optical character recognition model.

[0055] In another optional embodiment of the present application, before the above-mentioned first bill recognition result, the preset prompt word and the preset standard merchant information are used to perform model training on the preset large language model, it also includes:

[0056] The above-mentioned bill text recognition result is annotated with field type to obtain the annotated bill text recognition result.

[0057] Specifically, the designated fields in the above-mentioned bill text recognition results can be annotated according to the preset field types to obtain key bill fields, that is, fields with annotated field types. The above-mentioned preset field types may include "name type", "amount type", "time type", etc., and then the annotated bill text recognition results are uniformly recorded in a data table. Alternatively, the field types of the designated fields in the above-mentioned bill text recognition results can also be annotated according to the needs of the preset business analysis scenario, and then recorded in the data table. For example, assuming that the preset business analysis scenario is an automatic bill points scenario, the field types required for the automatic bill points scenario such as "store name", "time", and "amount" can be annotated in the above-mentioned bill text recognition results.

[0058] In the embodiment of the present application, the above-mentioned annotation method does not need to use annotation tools to annotate information such as the position and category of the text box in the image, but only needs to annotate and record the field type, which greatly reduces the annotation difficulty and annotation cost and improves data annotation efficiency.

[0059] Correspondingly, after the bill text recognition result is annotated, the first bill recognition result, the preset prompt word and the preset standard merchant information are used to perform model training on the preset large language model, including:

[0060] The preset large language model is trained using the annotated bill text recognition results, the preset prompt words and the preset standard merchant information.

[0061] Specifically, the above-mentioned preset prompt words can guide the preset large language model to extract field information such as merchant name from the field marked with the field type (i.e., the key bill field) during training, and then guide the above-mentioned preset large language model to map and match the extracted merchant name with the complete merchant name in the preset standard merchant information, until the training process meets the preset requirements and stops the training to obtain the trained bill recognition model.

[0062] In an embodiment of the present application, during model training, a preset large language model is guided by preset prompt words to extract field information such as merchant name from key bill fields that are labeled with field types, thereby reducing the amount of data processing during bill recognition model training, thereby improving the efficiency and accuracy of bill recognition model training.

[0063] In some embodiments, when the above-mentioned bill recognition model is applied, its reasoning process is often more complicated, resulting in the need for a large amount of computing resources and time. At the same time, the more complicated reasoning process will also affect the accuracy of the bill recognition model in bill recognition. Therefore, in order to improve the reasoning ability of the bill recognition model when applied, the above-mentioned preset prompt words include field extraction prompt words and field matching prompt words. The above-mentioned bill text recognition results after annotation, the above-mentioned preset prompt words and the above-mentioned preset standard merchant information are used to train the preset large language model, including:

[0064] Extracting field information from the annotated bill text recognition result according to the preset large language model and the field extraction prompt, wherein the field information corresponds to a field annotated with a field type in the bill text recognition result, and the field information at least includes a merchant name;

[0065] According to the preset large language model and the field matching prompt word, the merchant name in the field information is matched with the preset standard merchant information to obtain a merchant matching result;

[0066] The above-mentioned field information and the above-mentioned merchant matching results are used to perform model fine-tuning training on the above-mentioned preset large language model.

[0067] Among them, the above-mentioned field extraction prompt words are used to guide the preset large language model to extract the field information corresponding to the key bill field (i.e., the field marked with the field type) in the bill text recognition result, that is, to guide the extraction process of the field information, and the above-mentioned field matching prompt words are used to guide the preset large language model to match the extracted merchant name with the complete merchant name in the preset standard merchant information, that is, to guide the merchant name matching process. Finally, the weight of the above-mentioned preset large language model is fine-tuned according to the above-mentioned field information and the above-mentioned merchant matching results to obtain the adjusted bill recognition model. Of course, the above-mentioned field extraction prompt words and / or field matching prompt words can also constrain or guide the field output format, field matching range, etc. of the bill recognition model, which are not limited here.

[0068] Specifically, after the preset large language model is guided by the above-mentioned field extraction prompt words to extract field information from the bill text recognition result, since the merchant name in the bill image is often a font icon or a variant traditional Chinese character, etc., the merchant name in the field information may be incomplete or inaccurate. Therefore, the preset large language model is guided by the above-mentioned field matching prompt words to match the extracted merchant name and the complete merchant name, and a matching mapping relationship between "extracted merchant name" and "complete merchant name" can be established, thereby improving the accuracy of the bill recognition model in identifying the merchant name. In addition, through the above-mentioned field extraction prompt words and field matching prompt words, the reasoning process of the bill recognition model can be decomposed into step-by-step sub-problems, that is, the complex bill recognition process can be decomposed into a field extraction process and a field matching process, thereby reducing the complexity of the bill recognition problem, and then reducing the computing resources and processing time of the bill recognition model. At the same time, it can also avoid the confusion and errors that may occur when processing the bill recognition problem at one time, thereby improving the bill recognition ability of the bill recognition model.

[0069] In some embodiments, the field extraction prompt word includes a standard extraction field and an extraction range rule of the standard extraction field, the standard extraction field is a field output by the bill recognition model, the extraction range rule is used to determine the extraction range of the field information, and the field information is extracted from the annotated bill text recognition result according to the preset large language model and the field extraction prompt word, including:

[0070] Determine the target field type according to the extraction range rule of the standard extraction field, wherein the target field type is the field type corresponding to the standard extraction field in the annotated bill text recognition result;

[0071] The field information of the field marked with the target field type is extracted, and the extracted field information is used as the value corresponding to the standard extraction field.

[0072] Specifically, since the bill fields in different types of bill images may be quite different (for example, the code used to uniquely identify the bill image is expressed as an "order number" field in a certain type of bill image and as a "serial number" field in another type of bill image), in order to improve the generalization ability of the bill recognition model, one or more unified standard extraction fields can be set through the above-mentioned field extraction prompt words, and the extraction range rules corresponding to the standard extraction fields can be set. The target field type is determined through the extraction range rules of the above-mentioned standard extraction fields, and then the field information of the field marked with the above-mentioned target field type is used as the value corresponding to the standard extraction field. That is, the field information corresponding to the key bill fields that need to be extracted for each standard extraction field is further determined through the extraction range rules of the above-mentioned standard extraction fields, thereby improving the accuracy of field information extraction.

[0073] In the embodiment of the present application, the standard extraction fields included in the above-mentioned field extraction prompt words and the extraction range rules of the standard extraction fields can more accurately guide the preset large language model to extract field information.

[0074] In an optional embodiment, taking the ticket points scenario as an example, the preset prompt words may be:

[0075] Please follow the steps below to extract the information of the specified field from the OCR recognition result of the shopping receipt image and match the standard merchant name.

[0076] Step 1: Extract the specified 4 fields from the OCR recognition results.

[0077] 1. Extract the key "Merchant Name": Note that the merchant name may not have a field tag or annotation in the recognition result. It may be a logo icon or the name of the store;

[0078] 2. Extract key "receipt number": Please note that the receipt number refers to the unique voucher "order number" of the purchase, which may be marked or labeled as "order number", "serial number", "bill number", "No", etc.;

[0079] 3. Extract the key "Consumption Time": Note that the consumption time may be marked or labeled as "Checkout Date", "Checkout Time", "Order Time", etc. The output format of the consumption time is xxxx-xx-xx xx:xx:xx;

[0080] 4. Extract key "Consumption amount": Please note that the consumption amount refers to the actual transaction amount after deducting discounts and may be marked or labeled as "actual payment amount", "receipt amount", "consumption total", etc.;

[0081] Step 2: Match the "merchant name" extracted in step 1 from the standard merchant name list.

[0082] Note: If a match is found, change the value of "Merchant Name" to the standard merchant name. If a match is not found, change the value of "Merchant Name" to None.

[0083] Among them, the prompt word corresponding to step 1 is the above-mentioned field extraction prompt word. The "merchant name", "receipt number", "consumption time" and "consumption amount" in the field extraction prompt word are standard extraction fields. Each standard extraction field determines the field information to be extracted through the corresponding extraction range rule. For example, the extraction range rule corresponding to "receipt number" is "Note that the receipt number refers to the unique voucher "order number" for shopping, which may be marked or labeled as "order number", "serial number", "bill number", "No", etc.", indicating that the field information of key bill fields such as "order number", "serial number", "bill number", "No" needs to be extracted and uniformly used as the corresponding value of "receipt number".

[0084] For example, taking the bill points scenario as an example, assuming that the training data is saved in json format, each piece of data can contain an instruction field, an input field, and an output field expected by the model. Among them, the instruction field is the preset prompt word mentioned above; the input field can include the bill text recognition result, confidence, location information after OCR recognition and annotation (for example, [[[299.0,1005.0], [624.0,1015.0], [622.0,1065.0], [297.0,1055.0]], ('Amount payable: 38', 0.9)] ...), and preset standard merchant information (for example, a list of standard merchant names); the standard extraction fields and corresponding field information included in the output field can be as follows:

[0085] {

[0086] "Merchant Name":"XXX",

[0087] "Receipt Number":"114361273793878",

[0088] "Consumption time":"2024-03-20 18:58:56",

[0089] "Consumption amount":"38.0"

[0090] }.

[0091] It should also be noted that, when the input field includes the OCR-recognized and annotated bill text recognition results, confidence, location information, etc., the preset large language model can locate the field information more accurately based on the above location information. At the same time, since the above confidence is used to reflect the accuracy of the bill text recognition results, when the confidence is low (for example, less than the preset confidence threshold), the output field information can be set to "None" during training.

[0092] In some embodiments, the method of fine-tuning the preset large language model using the field information and the merchant matching result includes:

[0093] Calculate the loss value based on the above field information and the above merchant matching results;

[0094] When the above loss value meets the preset loss threshold, a fine-tuning model is obtained;

[0095] The weights of the fine-tuning model and the weights of the preset large language model are combined to obtain the weights corresponding to the bill recognition model.

[0096] Specifically, the above-mentioned preset large language model can be used as the basic model, and then the loss value is calculated according to the field information and merchant matching results output by the basic model, as well as the preset loss function. According to the changes in the loss value curve, the parameters of the model are adjusted accordingly, and then the new loss value is recalculated until the new loss value is less than or equal to the preset loss threshold, so as to obtain a fine-tuning model. Finally, the weight of the fine-tuning model and the weight of the basic model are merged, and the merged weight is the weight of the trained bill recognition model.

[0097] For example, the above-mentioned preset loss function can be one of the cross entropy loss function, negative log-likelihood loss, etc. Of course, other loss functions can also be used according to actual business needs, which is not limited here. The changes in the above-mentioned loss value curve include but are not limited to the following situations: the loss value curve is flat or almost does not decrease, the loss value curve converges quickly, and the loss value curve has abnormal values. Among them, when the loss value curve is flat or almost does not decrease, the learning rate can be increased; when the loss value curve converges quickly, the batch size can be reduced; when the loss value curve has abnormal values, the learning rate can be reduced.

[0098] In the embodiment of the present application, through fine-tuning training, the corresponding bill recognition model can be quickly trained based on the preset large language model.

[0099] Corresponding to the bill recognition model training method described in the above embodiment, Figure 2 A schematic diagram of a bill analysis method provided in an embodiment of the present application is shown, and is described in detail as follows:

[0100] S21. Acquire a bill image to be identified, perform image recognition on the bill image to be identified, and obtain a second bill recognition result.

[0101] Among them, taking the ticket points scenario as an example, the above-mentioned ticket image to be identified can be a shopping receipt image uploaded by the user through a terminal device (such as a mobile phone, computer, etc.). For example, the above-mentioned ticket image to be identified can be a photo of a paper receipt printed by a POS machine, a shopping screenshot of a program application, etc.

[0102] Specifically, after acquiring the above-mentioned bill image to be recognized, a preset optical character recognition (OCR) model can be used to quickly recognize the bill text, the confidence of the bill text, and the location information of the bill text in the bill image to be recognized.

[0103] S22. Using the second bill recognition result, a preset prompt word and preset standard merchant information as inputs of a bill recognition model, to obtain a bill recognition result output by the bill recognition model, wherein the preset standard merchant information at least includes a complete merchant name, the bill recognition model extracts at least a merchant name from the second bill recognition result according to the preset prompt word, and matches the extracted merchant name with the complete merchant name included in the preset standard merchant information according to the preset prompt word, and the bill recognition model is obtained according to the bill recognition model training method.

[0104] Among them, the above-mentioned preset standard merchant information can be a list of standard merchant names, and the list of standard merchant names can include the complete merchant names of all merchants in a supermarket or shopping platform. The above-mentioned preset prompt words refer to text or instructions that provide input to the bill recognition model to guide it to generate specific outputs. The above-mentioned preset prompt words can be used to constrain or guide the model execution capability of the bill recognition model, including but not limited to at least one of the following: model execution steps, model extraction range, model output range, model output format, etc. The bill recognition result output by the above-mentioned bill recognition model may include the complete merchant name, as well as other bill fields (for example, "receipt number", "consumption time", "consumption amount", etc.). The process of the above-mentioned bill recognition model outputting the bill recognition result according to the preset prompt words, the preset standard merchant information and the second bill recognition result can refer to the training process of the bill recognition model in the above embodiment, which will not be repeated here.

[0105] When identifying a bill image to be identified, the present application introduces preset prompt words and preset standard merchant information. The preset prompt words can guide the bill recognition model to extract at least the merchant name from the second bill recognition result, and guide the bill recognition model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information. The bill recognition model can match the complete merchant name, thereby avoiding the problem of incomplete recognition of the merchant name and improving the accuracy of bill recognition.

[0106] In some embodiments, the bill recognition result output by the bill recognition model can be used to perform bill analysis, and the bill analysis can include at least one of the following: automatic bill points, bill data analysis, user behavior analysis, user portrait analysis, etc. Assuming that in the automatic bill points scenario, the bill recognition result includes a merchant name recognition result, after obtaining the bill recognition result output by the bill recognition model, it also includes:

[0107] According to the above merchant name recognition result, the preset merchant database is matched to obtain a merchant name matching result;

[0108] If the merchant name matching result includes the target merchant name, then the to-be-recognized merchant image is subjected to a merchant analysis according to the merchant name recognition result and the merchant name matching result to obtain a merchant analysis result; the target merchant name is the merchant name matched according to the merchant name recognition result;

[0109] If the target merchant name is not included in the merchant name matching result, the process returns to the step of performing image recognition on the to-be-recognized bill image and subsequent steps, or issues an alarm.

[0110] The above merchant name recognition result may include the complete merchant name matched by the bill recognition model from the preset standard merchant information. Of course, since the bill image to be recognized uploaded by the user may be relatively blurry, it is impossible to accurately recognize and match the complete merchant name. Therefore, the above merchant name recognition result may be "none", that is, the merchant name is not recognized. The above preset merchant database may include merchant data of all merchants in the supermarket or shopping platform, such as merchant name, specific address and other information.

[0111] Specifically, the merchant name recognition result can be matched with the preset merchant database to obtain the merchant name matching result. If the above merchant name matching result includes the target merchant name, it can be determined whether the preset risk control rules are met. If the risk control rules are met, the bill recognition result and the merchant name matching result can be used to automatically score the bill. Of course, the matched target merchant name can also replace the product name in the merchant name matching result, and then the step of judging whether the preset risk control rules are met and the subsequent steps can be executed; if the risk control rules are not met, the step of performing image recognition on the bill image to be identified and the subsequent steps can be returned, or manual review and scoring can be performed through an alarm. At the same time, if the above merchant name matching result does not include the target merchant name, the step of performing image recognition on the bill image to be identified and the subsequent steps can also be returned, or manual review and scoring can be performed through an alarm.

[0112] Optionally, in the automatic bill points scenario, the above-mentioned preset risk control rules may include at least one of the following: whether the same bill is uploaded multiple times by different IDs, whether the number of points for the same ID on the day reaches the upper limit, whether the point amount of a single bill reaches the limit, etc.

[0113] It should also be noted that since erroneous information may appear in the bill recognition results, for example, errors in the recognition of the "consumption amount", which makes it impossible to accurately automatically score the bill, the bill recognition results can be reviewed before determining whether they meet the preset risk control rules. If there is an identification error, the consumption amount information can be manually modified and the points can be manually reviewed after the modification.

[0114] In an embodiment of the present application, since the merchant name recognition result may include a complete merchant name, a more accurate merchant name matching result may be obtained when the merchant name recognition result is matched with a preset merchant database, thereby improving the accuracy of automatic bill integration.

[0115] In order to better describe the automatic bill integration process, the following Figure 3 For explanation, refer to Figure 3First, obtain the bill image to be recognized uploaded by the user, and output the second bill recognition result through the OCR model, wherein the above-mentioned second bill recognition result may include bill text, confidence, location information, etc.; then use the second bill recognition result, the preset standard merchant name list, and the preset prompt word as the input of the bill recognition model to obtain the bill recognition result output by the bill recognition model, wherein the bill recognition result output by the bill recognition model may include: "merchant name", "receipt number", "consumption time", "consumption amount", etc.; then search the target merchant name in the preset merchant database according to the complete merchant name in the bill recognition result. If not found, manual review points are performed. If found, the found target merchant name can be used as the value corresponding to the "merchant name" in the bill recognition result; then determine whether the replaced bill recognition result is modified. If modified, manual review points are performed. If not modified, determine whether the preset risk control rules are met. If met, automatic bill points are performed. If not met, manual review points are performed.

[0116] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0117] Corresponding to the bill recognition model training method described in the above embodiment, Figure 4 A structural schematic diagram of a bill recognition model training device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0118] Reference Figure 4 The device may be a bill recognition model training device 41 , and the bill recognition model training device 41 may include a training data acquisition module 411 and a model training module 412 .

[0119] Reference Figure 4 The above-mentioned bill recognition model training device 41 comprises:

[0120] The training data acquisition module 411 is used to acquire a bill training set, perform image recognition on bill images in the bill training set, and obtain a first bill recognition result;

[0121] The model training module 412 is used to perform model training on the preset large language model using the above-mentioned first bill recognition result, the preset prompt word and the preset standard merchant information to obtain a bill recognition model; the above-mentioned preset standard merchant information at least includes a complete merchant name, and the above-mentioned preset prompt word is used to guide the above-mentioned preset large language model to extract at least the merchant name from the above-mentioned first bill recognition result, and guide the above-mentioned preset large language model to match the extracted merchant name with the complete merchant name included in the above-mentioned preset standard merchant information.

[0122] In some embodiments, when the training data acquisition module 411 performs image recognition on the bill image in the bill training set to obtain the first bill recognition result, it includes:

[0123] The preset optical character recognition model is used to perform text recognition on the bill images in the above-mentioned bill training set to obtain the above-mentioned first bill recognition result; the above-mentioned first bill recognition result at least includes a bill text recognition result.

[0124] In another optional embodiment of the present application, the above-mentioned bill recognition model training device 41 also includes a labeling module, which is used for the above-mentioned first bill recognition result, the preset prompt word and the preset standard merchant information to perform model training on the preset large language model, including:

[0125] The above-mentioned bill text recognition result is annotated with field type to obtain the annotated bill text recognition result.

[0126] Correspondingly, after the bill text recognition result is annotated, the model training module 412 performs model training on the preset large language model using the first bill recognition result, the preset prompt word and the preset standard merchant information, including:

[0127] The preset large language model is trained using the annotated bill text recognition results, the preset prompt words and the preset standard merchant information.

[0128] In some embodiments, when the above-mentioned bill recognition model is applied, its reasoning process is often more complicated, resulting in the need for a large amount of computing resources and time. At the same time, the more complicated reasoning process will also affect the accuracy of the bill recognition model in bill recognition. Therefore, in order to improve the reasoning ability of the bill recognition model when applied, the above-mentioned preset prompt words include field extraction prompt words and field matching prompt words. When the above-mentioned model training module 412 uses the above-mentioned bill text recognition results after annotation, the above-mentioned preset prompt words and the above-mentioned preset standard merchant information to train the preset large language model, it includes:

[0129] Extracting field information from the annotated bill text recognition result according to the preset large language model and the field extraction prompt, wherein the field information corresponds to a field annotated with a field type in the bill text recognition result, and the field information at least includes a merchant name;

[0130] According to the preset large language model and the field matching prompt word, the merchant name in the field information is matched with the preset standard merchant information to obtain a merchant matching result;

[0131] The above-mentioned field information and the above-mentioned merchant matching results are used to perform model fine-tuning training on the above-mentioned preset large language model.

[0132] In some embodiments, the field extraction prompt word includes a standard extraction field and an extraction range rule of the standard extraction field, the standard extraction field is a field output by the bill recognition model, and the extraction range rule is used to determine the extraction range of the field information. When the model training module 412 extracts the field information from the annotated bill text recognition result according to the preset large language model and the field extraction prompt word, it includes:

[0133] Determine the target field type according to the extraction range rule of the standard extraction field, wherein the target field type is the field type corresponding to the standard extraction field in the annotated bill text recognition result;

[0134] The field information of the field marked with the target field type is extracted, and the extracted field information is used as the value corresponding to the standard extraction field.

[0135] In some embodiments, when the model training module 412 uses the field information and the merchant matching result to perform model fine-tuning training on the preset large language model, it includes:

[0136] Calculate the loss value based on the above field information and the above merchant matching results;

[0137] When the above loss value meets the preset loss threshold, a fine-tuning model is obtained;

[0138] The weights of the fine-tuning model and the weights of the preset large language model are combined to obtain the weights corresponding to the bill recognition model.

[0139] Corresponding to the bill analysis method described in the above embodiment, Figure 5 A schematic diagram of the structure of a bill analysis device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0140] Reference Figure 5The device may be a bill analysis device 51 , and the bill analysis device 51 may include an image recognition module 511 and a bill recognition module 512 .

[0141] Reference Figure 5 The above-mentioned bill analysis device 51 comprises:

[0142] The image recognition module 511 is used to obtain a bill image to be recognized, perform image recognition on the bill image to be recognized, and obtain a second bill recognition result;

[0143] The bill recognition module 512 is used to use the above-mentioned second bill recognition result, the preset prompt word and the preset standard merchant information as inputs of the bill recognition model to obtain the bill recognition result output by the above-mentioned bill recognition model, wherein the above-mentioned preset standard merchant information at least includes a complete merchant name, the above-mentioned bill recognition model extracts at least the merchant name from the above-mentioned second bill recognition result according to the above-mentioned preset prompt word, and matches the extracted merchant name with the complete merchant name included in the above-mentioned preset standard merchant information according to the above-mentioned preset prompt word, and the above-mentioned bill recognition model is obtained according to the above-mentioned bill recognition model training method.

[0144] In some embodiments, the bill recognition result output by the bill recognition model can be used to perform bill analysis, and the bill analysis can include at least one of the following: automatic bill points, bill data analysis, user behavior analysis, user portrait analysis, etc. Assuming that in the automatic bill points scenario, the bill recognition result includes a merchant name recognition result, the bill analysis device 51 also includes a bill analysis module, which is used to, after obtaining the bill recognition result output by the bill recognition model, include:

[0145] According to the above merchant name recognition result, the preset merchant database is matched to obtain a merchant name matching result;

[0146] If the merchant name matching result includes the target merchant name, then the to-be-recognized merchant image is subjected to a merchant analysis according to the merchant name recognition result and the merchant name matching result to obtain a merchant analysis result; the target merchant name is the merchant name matched according to the merchant name recognition result;

[0147] If the target merchant name is not included in the merchant name matching result, the process returns to the step of performing image recognition on the to-be-recognized bill image and subsequent steps, or issues an alarm.

[0148] It should be noted that the information interaction, execution process, etc. between the devices / units are based on the same concept as the method embodiments of the present application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0149] Figure 6 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 6 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown in the figure), a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60. When the processor 60 executes the computer program 62, the steps in any of the method embodiments are implemented.

[0150] The electronic device 6 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 6 and does not constitute a limitation of the electronic device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include an input sending device, a network access device, a bus, etc.

[0151] The processor 60 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0152] In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. The memory 61 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 61 may also be used to temporarily store data that has been sent or is to be sent.

[0153] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the functional units and modules is used as an example. In practical applications, the function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0154] An embodiment of the present application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the method embodiments when executing the computer program.

[0155] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the various method embodiments can be implemented.

[0156] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the various method embodiments when executing the computer program product.

[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0158] In the embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0159] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0160] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0161] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A bill recognition model training method, characterized in that: include: Acquire a bill training set, perform image recognition on bill images in the bill training set, and obtain a first bill recognition result; The preset large language model is trained using the first bill recognition result, the preset prompt word and the preset standard merchant information to obtain a bill recognition model; the preset standard merchant information at least includes a complete merchant name, and the preset prompt word is used to guide the preset large language model to extract at least the merchant name from the first bill recognition result, and guide the preset large language model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information.

2. The bill recognition model training method according to claim 1, characterized in that: The performing image recognition on the bill images in the bill training set to obtain a first bill recognition result includes: The preset optical character recognition model is used to perform text recognition on the bill images in the bill training set to obtain the first bill recognition result; the first bill recognition result at least includes a bill text recognition result.

3. The bill recognition model training method according to claim 2, characterized in that: Before the method uses the first bill recognition result, the preset prompt word and the preset standard merchant information to perform model training on the preset large language model, it also includes: Annotating the bill text recognition result by field type to obtain an annotated bill text recognition result; The using the first bill recognition result, the preset prompt word and the preset standard merchant information to perform model training on the preset large language model includes: The preset large language model is trained using the annotated bill text recognition result, the preset prompt words and the preset standard merchant information.

4. The bill recognition model training method according to claim 3, characterized in that: The preset prompt words include field extraction prompt words and field matching prompt words, and the method of using the annotated bill text recognition result, the preset prompt words and the preset standard merchant information to perform model training on the preset large language model includes: Extracting field information from the annotated bill text recognition result according to the preset large language model and the field extraction prompt word, wherein the field information corresponds to a field annotated with a field type in the bill text recognition result, and the field information at least includes a merchant name; According to the preset large language model and the field matching prompt word, the merchant name in the field information is matched with the preset standard merchant information to obtain a merchant matching result; The preset large language model is fine-tuned using the field information and the merchant matching result.

5. The bill recognition model training method according to claim 4, characterized in that: The field extraction prompt word includes a standard extraction field and an extraction range rule of the standard extraction field, the standard extraction field is a field output by the bill recognition model, and the extraction range rule is used to determine the extraction range of the field information. The field information is extracted from the annotated bill text recognition result according to the preset large language model and the field extraction prompt word, including: Determining the target field type according to the extraction range rule of the standard extraction field, wherein the target field type is the field type corresponding to the standard extraction field in the annotated bill text recognition result; Field information of a field marked with the target field type is extracted, and the extracted field information is used as a value corresponding to the standard extraction field.

6. A bill analysis method, characterized in that: include: Acquire a bill image to be identified, perform image recognition on the bill image to be identified, and obtain a second bill recognition result; The second bill recognition result, a preset prompt word and preset standard merchant information are used as inputs of a bill recognition model to obtain a bill recognition result output by the bill recognition model, wherein the preset standard merchant information at least includes a complete merchant name, the bill recognition model extracts at least a merchant name from the second bill recognition result according to the preset prompt word, and matches the extracted merchant name with the complete merchant name included in the preset standard merchant information according to the preset prompt word, and the bill recognition model is obtained according to the bill recognition model training method according to any one of claims 1 to 5.

7. The bill analysis method according to claim 6, characterized in that: The bill recognition result includes a merchant name recognition result. After obtaining the bill recognition result output by the bill recognition model, the method further includes: Matching a preset merchant database according to the merchant name recognition result to obtain a merchant name matching result; If the merchant name matching result includes the target merchant name, then the to-be-recognized merchant image is subjected to a merchant analysis according to the merchant name recognition result and the merchant name matching result to obtain a merchant analysis result; the target merchant name is the merchant name matched according to the merchant name recognition result; If the target merchant name is not included in the merchant name matching result, the process returns to the step of performing image recognition on the to-be-recognized bill image and subsequent steps, or issues an alarm.

8. A bill recognition model training device, characterized in that: include: A training data acquisition module, used to acquire a bill training set, perform image recognition on bill images in the bill training set, and obtain a first bill recognition result; A model training module is used to perform model training on a preset large language model using the first bill recognition result, a preset prompt word and preset standard merchant information to obtain a bill recognition model; the preset standard merchant information at least includes a complete merchant name, and the preset prompt word is used to guide the preset large language model to extract at least the merchant name from the first bill recognition result, and guide the preset large language model to match the extracted merchant name with the complete merchant name included in the preset standard merchant information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 or claims 6 to 7 is implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, implements the method according to any one of claims 1 to 5 or claims 6 to 7.

Images