Computer vision-based shopping receipt parsing method and device, and storage medium

Abstract

The application provides a computer vision-based shopping receipt analysis method and device and a storage medium, comprising the following steps: obtaining a receipt text; the receipt text comprises text blocks composed of character information and position symbol information between the text blocks; converting the receipt text into a receipt image and recording first position information of each text block in the receipt text in the receipt image; inputting the receipt image into a trained computer vision model, identifying and outputting the category and corresponding second position information of each text block in the receipt image; and matching the first position information and the second position information to determine the character information corresponding to the text block of the category. The computer vision-based shopping receipt analysis method and device and the storage medium provided by the application can simultaneously mine text features and position features, and improve the accuracy and generalization of analysis.

Description

Technical Field

[0001] This invention relates to the field of information extraction, and in particular to a method, apparatus and storage medium for parsing shopping receipts based on computer vision. Background Technology

[0002] In recent years, artificial intelligence has gradually become an important driving force for global technological and industrial transformation, promoting the rise and development of business intelligence. Merchant receipt information analysis is a crucial application in the field of business intelligence, and it has gained widespread attention in recent years. Receipt analysis can aggregate order and transaction information from merchants in various shopping malls, shopping streets, and shopping centers, playing a vital role in mall operation analysis, mall business distribution, and merchant rental schemes.

[0003] Typical receipt parsing systems often design their parsing algorithms for specific receipts, making them unsuitable for the complex receipt formats encountered in real-world business scenarios. In practice, this leads to numerous parsing errors or failures to parse the receipts, significantly impacting downstream tasks (such as business analysis) and decision-making.

[0004] Existing technologies primarily rely on text matching rules. For example, if it's necessary to extract detailed product information (e.g., product name, unit price, quantity), a regular expression matching rule of "(Chinese character)[separator](number)[separator](number)" must be used. However, parsing errors occur when the "product name" contains other characters, or when the "unit price" and "quantity" contain unit characters. Furthermore, due to the wide variety of receipt styles, it's difficult to establish a unified matching rule. Summary of the Invention

[0005] To overcome the aforementioned technical deficiencies, this invention provides a method, apparatus, and storage medium for parsing shopping receipts based on computer vision. Specifically, the technical solution provided by this invention is as follows:

[0006] A computer vision-based method for parsing shopping receipts, comprising the following steps:

[0007] Obtain the receipt text; the receipt text contains text blocks composed of text information, as well as positional information between the text blocks;

[0008] The receipt text is converted into a receipt image, and the first position information of each text block in the receipt text in the receipt image is recorded.

[0009] The receipt image is input into a trained computer vision model to identify and output the category and corresponding second position information of each text block in the receipt image;

[0010] The first location information is matched with the second location information to determine the text information corresponding to the text block of the category.

[0011] In some implementations, converting the receipt text into a receipt image specifically includes:

[0012] Determine the text parameters and canvas parameters;

[0013] Based on the location information in the receipt text, the receipt text is decomposed into multiple text blocks and the size of each text block after being converted into an image is calculated; the location information includes horizontal location symbols and vertical location symbols;

[0014] The horizontal spacing between text blocks is determined based on the number of horizontal locators between them;

[0015] The vertical spacing between text blocks is determined based on the number of vertical locators between them.

[0016] Based on the size of each text block in the receipt, as well as the horizontal and vertical spacing between the text blocks, the receipt text is rendered onto the canvas to generate a receipt image.

[0017] In some implementations, the computer vision-based shopping receipt parsing method further includes:

[0018] Training the computer vision model; specifically including:

[0019] Capture receipt images;

[0020] Label the category and location information of each text block in the receipt image; the category includes any combination of one or more of the following: product name, unit price, quantity, amount, order number, and total order amount;

[0021] The labeled receipt images are used as samples in the training set;

[0022] The computer vision model is trained using samples from the training set until the number of training iterations reaches a preset number of iterations or the objective function curve of the computer vision model converges, at which point the training ends.

[0023] In some implementations, matching the first location information with the second location information to determine the text information corresponding to the text block of the category specifically includes:

[0024] Based on the user's target category instruction and the recognition output of the computer vision model, the second location information of the target category is determined;

[0025] Find the first location information that matches the second location information, and obtain the text information of the target text block corresponding to the matching first location information;

[0026] Extract the text information of the target category and the corresponding target text block.

[0027] In some implementations, the step of using the receipt image as an input sample and inputting it into a trained computer vision model to detect and obtain the second location information and category of the target information includes:

[0028] After obtaining the output of the computer vision model, non-maximum suppression (NMS) is used for post-processing to remove overlapping or text block position information that does not contain the target information.

[0029] The present invention also provides a computer vision-based shopping receipt parsing device, comprising:

[0030] The acquisition module is used to acquire the receipt text; the receipt text contains text blocks composed of text information, as well as position information between each text block;

[0031] The receipt imaging module is used to convert the receipt text into a receipt image and record the first position information of each text block in the receipt text in the receipt image.

[0032] The image parsing module is used to input the receipt image into a trained computer vision model, identify and output the category and corresponding second position information of each text block in the receipt image;

[0033] The location matching module is used to match the first location information with the second location information to determine the text information corresponding to the text block of the category.

[0034] In some embodiments, the receipt imaging module specifically includes:

[0035] The parameter determination submodule is used to determine text parameters and canvas parameters;

[0036] The location character recognition submodule is used to recognize location character information in the receipt text; the location character information includes horizontal location characters and vertical location characters.

[0037] The text decomposition submodule is used to decompose the receipt text into multiple text blocks based on the positional information in the receipt text and to calculate the size of each text block after it is converted into an image.

[0038] The spacing determination submodule is used to determine the horizontal spacing between text blocks based on the number of horizontal locators between them; and to determine the vertical spacing between text blocks based on the number of vertical locators between them.

[0039] The rendering submodule is used to render the receipt text onto the canvas according to the size of each text block in the receipt, as well as the horizontal and vertical spacing between the text blocks, to generate a receipt image.

[0040] The information storage submodule is used to store the text information of each text block and the first position information corresponding to the text block.

[0041] In some embodiments, the computer vision-based shopping receipt parsing device further includes:

[0042] A model training module is used to train the computer vision model; the model training module specifically includes:

[0043] The sample acquisition submodule is used to acquire receipt images;

[0044] The annotation submodule is used to annotate the category and location information of each text block in the receipt image. The category includes any combination of one or more of the following: product name, unit price, quantity, amount, order number, and total order amount.

[0045] The training submodule is used to put the labeled ticket images into the training set as samples, and train the computer vision model using the samples in the training set until the number of training iterations reaches the preset number of iterations or the objective function curve of the computer vision model converges, at which point the training ends.

[0046] In some implementations, the location matching module specifically includes:

[0047] The second position determination submodule is used to determine the second position information of the target category based on the user's target category instruction and the recognition output of the computer vision model.

[0048] The matching submodule is used to find the first location information that matches the second location information and to obtain the text information of the target text block corresponding to the matched first location information;

[0049] The information extraction submodule is used to extract the text information of the target category and the corresponding target text block.

[0050] The present invention also provides a storage medium storing computer program code, which, when executed by a processor, implements the computer vision-based shopping receipt parsing method as described above.

[0051] Compared with existing technologies, the computer vision-based shopping receipt parsing method, device, and storage medium proposed in this invention have at least the following beneficial effects:

[0052] 1. The computer vision-based shopping receipt parsing method, device, and storage medium provided by this invention match the position information obtained during receipt text modeling and imaging with the position information obtained after image recognition. This allows for the rapid determination of the text block position information under the target category. Combined with the text information during modeling, it enables the rapid acquisition of text information under different categories in the required receipt information, such as order number, total amount, and product name. The target text information parsing accuracy is high, interference information is low, and parsing efficiency is significantly improved.

[0053] 2. In the solution of the present invention, the receipt text obtained by location matching is the original text information. Compared with the prior art technology of redesigning the text recognition algorithm in the image, the result extracted by the present invention is more accurate.

[0054] 3. This invention constructs a receipt image based on the receipt text, and then matches the text information of the corresponding text block through visual pattern recognition location information. It can simultaneously mine text features and location features, and is applicable to shopping receipts of various styles. Therefore, this invention has extremely high versatility. Attached Figure Description

[0055] The preferred embodiments will now be described in a clear and easy-to-understand manner, with reference to the accompanying drawings, to further explain the above-mentioned characteristics, technical features, advantages, and implementation methods of a computer vision-based shopping receipt parsing method, apparatus, and storage medium.

[0056] Figure 1 This is a flowchart of an embodiment of the computer vision-based shopping receipt parsing method of the present invention;

[0057] Figure 2 This is a schematic diagram of the receipt text in another embodiment of the computer vision-based receipt parsing method of the present invention;

[0058] Figure 3 This is a schematic diagram of the receipt imaging result in another embodiment of the computer vision-based receipt parsing method of the present invention;

[0059] Figure 4 This is a schematic diagram of the information extraction results in another embodiment of the computer vision-based shopping receipt parsing method of the present invention;

[0060] Figure 5 This is a structural block diagram of an embodiment of the computer vision-based shopping receipt parsing device of the present invention;

[0061] Figure 6 This is a structural block diagram of another embodiment of the computer vision-based shopping receipt parsing device of the present invention;

[0062] Figure 7This is a structural block diagram of another embodiment of the computer vision-based shopping receipt parsing device of the present invention;

[0063] Figure 8 This is a structural block diagram of another embodiment of the computer vision-based shopping receipt parsing device of the present invention. Detailed Implementation

[0064] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0065] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or sets.

[0066] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0067] Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0069] In one embodiment, reference Figure 1 As shown, the basic process of the computer vision-based shopping receipt parsing method provided by this invention is as follows:

[0070] S100, retrieve receipt text;

[0071] S200, convert the receipt text into a receipt image, and record the first position information of each text block in the receipt text in the receipt image;

[0072] S300, the receipt image is input into the trained computer vision model to identify and output the category and corresponding second position information of each text block in the receipt image;

[0073] S400, match the first location information with the second location information to determine the text information corresponding to the text block of the category.

[0074] Specifically, the acquired receipt text consists of textual and positional information. The information in the receipt text is categorized and labeled to form text blocks. Then, a receipt image is generated on a canvas, recording the positional and textual information of each text block. The generated receipt image is input into a trained computer vision model. This model can output the category and positional information of each text block based on the input receipt image, such as order number text blocks, total amount text blocks, and product name text blocks. The obtained positional information is matched with the positional information stored during receipt imaging to obtain the category and corresponding textual information of the text block. Compared to existing technologies, this process achieves higher accuracy in obtaining textual information, thus enabling the parsing and processing of shopping receipts.

[0075] In some implementations, based on the above embodiments, the step of converting the receipt text into a receipt image and recording the first position information of each text block in the receipt text within the receipt image includes:

[0076] Determine the text parameters and canvas parameters;

[0077] Based on the location information in the receipt text, the receipt text is decomposed into multiple text blocks and the size of each text block after being converted into an image is calculated; the location information includes horizontal location symbols and vertical location symbols;

[0078] The horizontal spacing between text blocks is determined based on the number of horizontal locators between them;

[0079] The vertical spacing between text blocks is determined based on the number of vertical locators between them.

[0080] Based on the size of each text block in the receipt, as well as the horizontal and vertical spacing between the text blocks, the receipt text is rendered onto the canvas to generate a receipt image.

[0081] Specifically, text parameters and canvas parameters are determined based on the receipt text. Text parameters include font, font color, font weight, and slant, while canvas parameters include canvas size and background color. Commonly used delimiters on receipts are "\t" for horizontal spacing and "\n" for vertical spacing. The receipt text information is as follows: Figure 2 As shown, after imaging... Figure 3 As shown, the top left corner of the receipt image can be set as the origin of the coordinate system, and the text information and position information of each text block can be represented and stored using the horizontal and vertical coordinates.

[0082] In some implementations, based on the above embodiments, the following steps are also included:

[0083] S000, train the computer vision model to output the position information of each text block and the category of each text block based on the receipt image. The specific process is as follows:

[0084] Capture receipt images;

[0085] Label the category and location information of each text block in the receipt image; the category includes any combination of one or more of the following: product name, unit price, quantity, amount, order number, and total order amount;

[0086] The labeled receipt images are used as samples in the training set;

[0087] The computer vision model is trained using samples from the training set until the number of training iterations reaches a preset number of iterations or the objective function curve of the computer vision model converges, at which point the training ends.

[0088] Specifically, the text block categories are manually assigned. Unnecessary categories, such as the store greeting "Thank you for your patronage, welcome to visit again," can be assigned to other categories. The labeled receipt images are used as the training sample set and input into the computer vision model for training. The model takes a receipt image as input and outputs the text block location coordinates and their respective categories. Using the receipt image as the input sample and location information and category as the optimization objectives, the model is optimized through backpropagation until the objective function of the computer vision model converges, at which point training ends. On the test set, the classification effect of the computer vision model is judged by accuracy, and the detection effect of the second location information is judged by recall and precision. The feature extraction of the model can use ResNet, which has good performance, and the Feature Pyramid Network (FPN) is used to fuse features of different sizes.

[0089] In some implementations, based on the above embodiments, matching the first location information with the second location information to determine the text information corresponding to the text block of the category specifically includes:

[0090] Based on the user's target category instruction and the recognition output of the computer vision model, the second location information of the target category is determined;

[0091] Find the first location information that matches the second location information, and obtain the text information of the target text block corresponding to the matching first location information;

[0092] Extract the text information of the target category and the corresponding target text block.

[0093] Specifically, for example, if you need categories such as order number, order time, order amount, payment method, and shopping details, the shopping details need to include three categories: item name, unit price, and quantity. This can be achieved by comparing the output of the computer vision model (e.g., category: order number, location (x2, y2)) with the location information in the receipt image from step S200, and then obtaining the text information of the matching location information text block, for example (location (x1, y1), order number 36471387492834523). This allows you to obtain the accurate category and corresponding text information. Other required categories can be obtained in the same way. The final formatted output result is shown in the reference. Figure 4 As shown, the Intersection over Union (IOU) measurement standard can be used in the location matching process. For example, when the IOU is greater than 50%, the match is considered successful. The advantages of this approach are high accuracy in text information parsing, less interference information, and significantly improved parsing efficiency.

[0094] In some implementations, based on the above embodiments, after the step of inputting the receipt image into a trained computer vision model to identify and output the category and corresponding second position information of each text block in the receipt image, the method further includes:

[0095] Non-maximum suppression (NMS) is used to post-process the recognition output of the computer vision model to remove overlapping or text block position information outside of pre-defined categories.

[0096] Specifically, conventional post-processing methods can be used here, such as non-maximum suppression (NMS) to remove overlapping or text block position information outside of pre-defined categories, making the matching results of the target information more accurate.

[0097] Based on the same technical concept, the present invention also provides a shopping receipt parsing device based on computer vision, for reference. Figure 5 As shown, it includes:

[0098] The acquisition module is used to acquire the receipt text; the receipt text contains text blocks composed of text information, as well as position information between each text block;

[0099] The receipt imaging module is used to convert the receipt text into a receipt image and record the first position information of each text block in the receipt text in the receipt image.

[0100] The image parsing module is used to input the receipt image into a trained computer vision model, identify and output the category and corresponding second position information of each text block in the receipt image;

[0101] The location matching module is used to match the first location information with the second location information to determine the text information corresponding to the text block of the category.

[0102] Specifically, this embodiment uses a cash register and a receipt printer, applicable to scenarios such as supermarkets, farmers' markets, law enforcement, industry, and logistics. The acquisition module acquires the text information of the shopping receipt, then the receipt imaging module divides the text information into individual text blocks. Next, the receipt text is imaged, and the location and text information corresponding to each text block are recorded. The image parsing module calls a computer vision model to parse the receipt text image, outputting the category and location information corresponding to each text block. Finally, the location matching module matches the two sets of location information to obtain the information needed by the user.

[0103] In some implementations, based on the above embodiments, reference is made to... Figure 6 The receipt imaging module specifically includes:

[0104] The parameter determination submodule is used to determine text parameters and canvas parameters;

[0105] The location character recognition submodule is used to recognize location character information in the receipt text; the location character information includes horizontal location characters and vertical location characters.

[0106] The text decomposition submodule is used to decompose the receipt text into multiple text blocks based on the positional information in the receipt text and to calculate the size of each text block after it is converted into an image.

[0107] The spacing determination submodule is used to determine the horizontal spacing between text blocks based on the number of horizontal locators between them; and to determine the vertical spacing between text blocks based on the number of vertical locators between them.

[0108] The rendering submodule is used to render the receipt text onto the canvas according to the size of each text block in the receipt, as well as the horizontal and vertical spacing between the text blocks, to generate a receipt image.

[0109] The information storage submodule is used to store the text information of each text block and the first position information corresponding to the text block.

[0110] Specifically, in the parameter determination submodule, the text parameters to be determined include font type, font color, font slant, and font thickness, while the canvas parameters to be determined include canvas size and canvas background color. The text decomposition submodule and the spacing determination submodule are used to parse the receipt text information. Then, the parsed data is rendered on the canvas with the determined parameters through the rendering imaging submodule to generate the receipt image. Finally, the information storage submodule stores the position information and text information corresponding to each text block.

[0111] In some implementations, based on the above embodiments, reference is made to... Figure 7 It also includes:

[0112] A model training module is used to train the computer vision model; the model training module specifically includes:

[0113] The sample acquisition submodule is used to acquire receipt images;

[0114] The annotation submodule is used to annotate the category and location information of each text block in the receipt image. The category includes any combination of one or more of the following: product name, unit price, quantity, amount, order number, and total order amount.

[0115] The training submodule is used to put the labeled ticket images into the training set as samples, and train the computer vision model using the samples in the training set until the number of training iterations reaches the preset number of iterations or the objective function curve of the computer vision model converges, at which point the training ends.

[0116] Specifically, the sample collection submodule collects receipt images needed to train the computer vision model, and then the annotation submodule annotates the collected receipt images, such as total amount, order number, unit price and quantity. After the training sample set is improved, the annotated receipt images are input into the computer vision model for training until the computer vision model can achieve the expected effect or the number of iterations reaches the preset number before it is put into use.

[0117] In some implementations, based on the above embodiments, reference is made to... Figure 8 The location matching module specifically includes:

[0118] The second position determination submodule is used to determine the second position information of the target category based on the user's target category instruction and the recognition output of the computer vision model.

[0119] The matching submodule is used to find the first location information that matches the second location information and to obtain the text information of the target text block corresponding to the matched first location information;

[0120] The information extraction submodule is used to extract the text information of the target category and the corresponding target text block.

[0121] Specifically, the matching submodule matches the position information output by the computer vision model with the position information of the text blocks stored when the receipt image is generated. After finding the text information of the corresponding text block, the information extraction submodule extracts the above information and then organizes it into a formatted text information for output.

[0122] In one embodiment of the present invention, a storage medium stores at least one instruction, which is loaded and executed by a processor to perform the operations described in the corresponding embodiments of the computer vision-based shopping receipt parsing method, apparatus, and storage medium. For example, the storage medium may be a read-only memory (ROM), random access memory (RAM), read-only optical disc (CD-ROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0123] These can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, this invention is not limited to any particular combination of hardware and software.

[0124] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0125] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0126] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] Furthermore, the functional units in the various embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0128] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A computer vision based method for parsing a shopping receipt, the method comprising: The method comprises the steps of: obtaining a receipt text; the receipt text comprises text blocks composed of character information and position symbol information between the text blocks; converting the receipt text into a receipt image and recording first position information of each text block in the receipt text in the receipt image; inputting the receipt image into a trained computer vision model, identifying and outputting the category and corresponding second position information of each text block in the receipt image; matching the first position information and the second position information to determine the character information corresponding to the text block of the category; wherein the conversion of the receipt text into the receipt image comprises: determining text parameters and canvas parameters; decomposing the receipt text into multiple text blocks according to the position symbol information in the receipt text and calculating the size of each text block after being converted into an image; the position symbol information comprises horizontal position symbols and vertical position symbols; determining the horizontal spacing between the text blocks according to the number of horizontal position symbols between the text blocks; determining the vertical spacing between the text blocks according to the number of vertical position symbols between the text blocks; rendering the receipt text onto a canvas according to the size of each text block in the receipt, the horizontal spacing and the vertical spacing between the text blocks to generate a receipt image.

2. The computer vision based shopping slip parsing method of claim 1, wherein, Further comprising: training the computer vision model; specifically comprising: collecting receipt images; labeling the category and position information of each text block in the receipt image; the category comprises one or more of any combination of product name, unit price, quantity, amount, order number and total order amount; putting the labeled receipt image into a training set as a sample; training the computer vision model through the samples in the training set until the number of training iterations reaches a preset iteration number or the objective function curve of the computer vision model converges.

3. The computer vision based shopping slip parsing method of claim 1, wherein, The matching of the first position information and the second position information to determine the character information corresponding to the text block of the category specifically comprises: determining the second position information of the target category according to the target category instruction of the user and the recognition output result of the computer vision model; finding the first position information matched with the second position information and obtaining the character information of the target text block corresponding to the matched first position information; extracting the target category and the character information of the corresponding target text block.

4. The computer vision based shopping slip parsing method of claim 1, wherein, After the computer vision model identifies and outputs the category and corresponding second position information of each text block in the receipt image, further comprising: using non-maximum suppression (NMS) to post-process the recognition output result of the computer vision model to remove the text block position information other than the overlapping or pre-set category.

5. A computer vision based slip parsing apparatus, characterized by, comprising: an acquisition module for acquiring a receipt text; the receipt text comprises text blocks composed of character information and position symbol information between the text blocks; a receipt imaging module for converting the receipt text into a receipt image and recording first position information of each text block in the receipt text in the receipt image; An image analysis module is configured to input the receipt image into the trained computer vision model, identify and output the category and corresponding second position information of each text block in the receipt image. A position matching module is configured to match the first position information with the second position information, and determine the character information of the text block of the category. The receipt imaging module specifically includes: A parameter determination sub-module is configured to determine text parameters and canvas parameters. A position symbol identification sub-module is configured to identify position symbol information in the receipt text, wherein the position symbol information includes horizontal position symbols and vertical position symbols. A text decomposition sub-module is configured to decompose the receipt text into multiple text blocks according to the position symbol information in the receipt text, and calculate the size of each text block after being converted into an image. A distance determination sub-module is configured to determine the horizontal distance between text blocks according to the number of horizontal position symbols between the text blocks, and determine the vertical distance between text blocks according to the number of vertical position symbols between the text blocks. A rendering imaging sub-module is configured to render the receipt text onto a canvas according to the size of each text block in the receipt, and the horizontal distance and vertical distance between the text blocks, to generate a receipt image. An information storage sub-module is configured to store the text information of each text block and the first position information corresponding to the text block.

6. The computer vision based shopping ticket parsing apparatus of claim 5, wherein, Further comprising: A model training module is configured to train the computer vision model. The model training module specifically includes: A sample collection sub-module is configured to collect receipt images. A labeling sub-module is configured to label the category and position information of each text block in the receipt image, wherein the category includes one or more of any combination of product name, unit price, quantity, amount, order number, and total order amount. A training sub-module is configured to put the labeled receipt image into a training set as a sample, train the computer vision model through the sample in the training set, and end the training when the number of training iterations reaches a preset iteration number or the objective function curve of the computer vision model converges.

7. The computer vision based shopping ticket parsing apparatus of claim 5, wherein, The position matching module specifically includes: A second position determination sub-module is configured to determine the second position information of the target category according to the target category instruction of the user and the identification output result of the computer vision model. A matching sub-module is configured to find the first position information matching the second position information, and obtain the character information of the target text block corresponding to the matched first position information. An information extraction sub-module is configured to extract the character information of the target category and the target text block.

8. A storage medium storing computer program code, which, when executed by a processor, implements the computer vision-based receipt analysis method according to any one of claims 1-4.

Images