Terminal device-based signed receipt information identification method, device, medium and equipment

Abstract

The application discloses a terminal equipment-based signed receipt information identification method and device, medium and equipment. The method comprises the following steps: obtaining a signed receipt image, identifying the current angle of the signed receipt image through an SVM network, and rotating the signed receipt to a standard angle according to the current angle; performing verification element detection on the signed receipt image by using a YOLO network, determining the bounding box position and category of the verification element, and identifying whether the bounding box position and category of each verification element in the signed receipt image match; if the bounding box position and category match, each verification element is cropped according to the bounding box position; the verification element is input into a text recognition model for text recognition; and if the text recognition result matches the signed receipt information recorded in advance, it is determined that the information identification of the signed receipt is completed. According to the technical scheme, main verification matching processing can be performed on the mobile equipment side, and in the case that the verification is unqualified, the mobile equipment needs to inform the business party of the unqualified verification element and the reason for the unqualification.

Description

Technical Field

[0001] This application relates to the field of logistics technology, and in particular to a method, device, medium and equipment for identifying receipt information based on terminal equipment. Background Technology

[0002] In the current logistics industry, many tasks still require manual labor. For example, verifying signed delivery slips.

[0003] During the verification process, the content to be verified often includes whether there is a signature, whether there is a stamp, and whether the stamp is consistent with the registered stamp, such as the shape, text and code of the stamp. In addition, in some scenarios, it is also necessary to verify whether the ID number of the signatory is consistent with the registered ID number, and whether the products listed in the receipt are specific types of products.

[0004] However, manual verification not only carries a certain probability of omissions, but also has low efficiency. Therefore, overcoming the problems of low efficiency, high complexity, and risk of omissions in the verification of signed receipts has become a technical challenge in this field. Summary of the Invention

[0005] This application provides a method, apparatus, medium, and device for identifying signed receipt information based on a terminal device. This solution uses a mobile device to photograph the signed receipt. The shape and location of the stamp, as well as the locations of the ID number, date, and product name, are then verified and matched on the mobile device. If the verification fails, the business party is notified via the mobile device of the failed verification elements and the reason for the failure.

[0006] This application provides a method for recognizing signed receipt information based on a terminal device. The method is executed by the terminal device and includes:

[0007] The system acquires a signed receipt image, identifies the current angle of the signed receipt image using an SVM network, and generates a rotation command to rotate the signed receipt image to a standard angle if the current angle is not a standard angle.

[0008] The YOLO network is used to detect verification elements in the signed receipt image, determine the border position and category of the verification elements, and identify whether the border position and category of each verification element in the signed receipt image match.

[0009] If a match is found, each verification element is cropped according to the border position.

[0010] The cropped verification elements are input into the text recognition model for text recognition.

[0011] If the text recognition result matches the pre-registered receipt information, then the information recognition of the receipt is considered complete.

[0012] Furthermore, after using the YOLO network to detect verification elements in the signed receipt image and determine the border position and category of the verification elements, the method further includes:

[0013] Identify whether the category of the verification element includes the category of verification elements identified by the signed receipt information;

[0014] If not, a message indicating that the signature information recognition failed will be generated.

[0015] Further, the system acquires a signed receipt image, identifies the current angle of the signed receipt image using an SVM network, and if the current angle is not a standard angle, generates a rotation command to rotate the signed receipt image to a standard angle, including:

[0016] Scan the receipt to obtain the original image of the receipt, normalize the original image of the receipt, and convert the original image of the receipt into a uniform and fixed height and width as the receipt image;

[0017] The pixels of the signed receipt image are downsampled and input into the SVM network to determine whether the current angle of the signed receipt is positive.

[0018] If the receipt is not in the positive direction, prompt the recipient to rescan the receipt.

[0019] Furthermore, the YOLO network is used to detect verification elements in the signed receipt image, determine the border position and category of the verification elements, and identify whether the border position and category of each verification element in the signed receipt image match, including:

[0020] The signed receipt image is input into the YOLO network to obtain the border position and category of the verification element in the signed receipt image;

[0021] The system identifies whether the border position and category of the verification element are within a preset verification element area. If the border position of the verification element is within the preset verification element area and the category of the verification element corresponds to the preset verification element area, then the verification element is identified as existing. If the border position of the verification element is not within the preset verification element area and the category of the verification element does not correspond to the preset verification element area, then the verification element is identified as not existing.

[0022] Furthermore, the text recognition model training phase includes:

[0023] Acquire a preset number of curved text images, machine-printed text images, and handwritten text images;

[0024] The curved text image is scaled, cropped, mirrored, and rotated proportionally; the machine-printed text image is scaled, cropped, and mirrored proportionally; and the handwritten text image is scaled, cropped, and mirrored proportionally to obtain sample data.

[0025] A character recognition model is trained based on the sample data.

[0026] Furthermore, the method also includes:

[0027] The text recognition model is stored in a preset storage location on the terminal device, so that after each verification element is cropped according to the border position, the text recognition model can be called.

[0028] This application provides a terminal device for recognizing signed receipt information, the device being configured on the terminal device, the device comprising:

[0029] An angle processing unit is used to acquire a signed receipt image, identify the current angle of the signed receipt image through an SVM network, and generate a rotation command to rotate the signed receipt image to a standard angle if the current angle is not a standard angle.

[0030] The verification element detection unit is used to perform verification element detection in the signed receipt image using the YOLO network, determine the border position and category of the verification element, and identify whether the border position and category of each verification element in the signed receipt image match.

[0031] The verification element cropping unit is used to crop each verification element according to the border position if a match is found.

[0032] The text recognition unit is used to input the cropped verification elements into the text recognition model for text recognition.

[0033] The information recognition result determination module is used to determine that the information recognition of the receipt is complete if the text recognition result matches the pre-registered receipt information.

[0034] Furthermore, the device also includes an element category matching verification unit, used for:

[0035] Identify whether the category of the verification element includes the category of verification elements identified by the signed receipt information;

[0036] If not, a message indicating that the signature information recognition failed will be generated.

[0037] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the terminal device-based signature information identification method as described in this application.

[0038] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the terminal device-based signature information identification method as described in this application.

[0039] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0040] The technical solution provided in this application simplifies the computation of determining the direction of the receipt by using SVM, transforming the original second-order recognition network into a first-order one. Furthermore, since the format of the receipt is relatively fixed, after detecting the category of the verification element, the category to which the verification element belongs can be quickly determined based on its location, simplifying the calculation of the confidence score of the verification element category. Moreover, during the training phase, a large number of customized stamps and machine-printed text are introduced, greatly improving the scalability of the model. When a new stamp is added, only the text of the new stamp needs to be trained, shortening the training time. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0042] Figure 1 This is a flowchart illustrating the terminal device-based signature information identification method provided in Embodiment 1 of this application;

[0043] Figure 2 This is a schematic diagram of the structure of the automatic receipt recognition system provided in the embodiments of this application;

[0044] Figure 3 This is a schematic diagram of the process for recognizing receipt information based on a terminal device, as provided in an embodiment of this application.

[0045] Figure 4 This is a schematic diagram of the structure of the receipt information recognition device based on terminal equipment provided in Embodiment 2 of this application;

[0046] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0049] Example 1

[0050] Figure 1 This is a flowchart illustrating the terminal device-based signature information recognition method provided in Embodiment 1 of this application. This embodiment is applicable to situations where signatures are quickly verified using mobile terminal devices. This method can be executed by the terminal device-based signature information recognition device provided in this embodiment. The device can be implemented in software and / or hardware and can be integrated into an electronic device for terminal device-based signature information recognition.

[0051] like Figure 1 As shown, the method includes:

[0052] S110. Obtain the signed receipt image, identify the current angle of the signed receipt image through an SVM network, and if the current angle is not a standard angle, generate a rotation command to rotate the signed receipt image to a standard angle.

[0053] The signed receipt image can be captured by the client using their mobile phone camera. Elements of the signed receipt image can include a stamp, the customer's handwritten signature, the order number, and the date. In this solution, the parsing method can be based on image recognition technology. For example, it can identify the red text as a stamp, the text in the signature area as the customer's handwritten signature, the barcode or QR code as the order number, and the date information corresponding to the printed date on the signed receipt. Based on the distribution of the above information, the current angle of the signed receipt is determined.

[0054] In this solution, the standard angle can be 0°. At this standard angle, it facilitates subsequent tasks such as recognizing elements on the delivery receipt in the image. By setting it up this way, this solution simplifies the computation on the terminal device through the SVM network, while simultaneously improving the efficiency and accuracy of delivery receipt recognition by rotating it to the standard angle.

[0055] In this embodiment, optionally, the original image of the receipt is obtained by scanning the receipt, the original image of the receipt is normalized, and the original image of the receipt is converted into a uniform and fixed height and width as the receipt image.

[0056] The pixels of the signed receipt image are downsampled and input into the SVM network to determine whether the current angle of the signed receipt is positive.

[0057] If the receipt is not in the positive direction, prompt the recipient to rescan the receipt.

[0058] This solution further downsamples the preprocessed image pixels to obtain a compressed image, reducing the image data size. The compressed image is then used to calculate the angle of the image using an SVM, which reduces the computational load compared to using a CNN to determine the direction of the delivery slip.

[0059] S120. Use the YOLO network to detect verification elements in the signed receipt image, determine the border position and category of the verification elements, and identify whether the border position and category of each verification element in the signed receipt image match.

[0060] The YOLO network can be a pre-trained neural network model.

[0061] S130. If a match is found, each verification element is cropped according to the border position.

[0062] S140. Input the cropped verification elements into the text recognition model for text recognition.

[0063] Among them, OCR can be used to recognize curved text, handwritten text, and machine-printed text obtained by the key area cropping submodule of the verification element acquisition module.

[0064] Curved text corresponds to the text included in the company's official seal when signing for receipt, such as "XXX Technology Company," etc. Handwritten text corresponds to the name of the company or individual signing for receipt; handwritten text can be categorized and recognized based on different fonts. Machine-printed text refers to the text included in the machine-printed information on the receipt.

[0065] S150. If the text recognition result matches the pre-registered receipt information, then the information recognition of the receipt is determined to be complete.

[0066] The pre-registered delivery receipt information can be the recipient's name, ID number, etc., stored in the system, or the name and identification code of the receiving unit, etc. After collecting the corresponding delivery receipt elements, the border position of the delivery receipt elements can be verified against their category to obtain a matching result.

[0067] For example, if the border position and type are both matched, then the current receipt element is determined to meet the requirements, the recognition is complete, and the recognition of the next receipt element is performed. When the recognition of all receipt elements is completed, the information recognition of the receipt can be determined to be complete.

[0068] Figure 2 This is a schematic diagram of the structure of the automatic receipt recognition system provided in this application embodiment. For example... Figure 2 As shown, the system includes:

[0069] Orientation rotation module: Used to determine the image orientation, identifying whether the delivery receipt is oriented correctly via an SVM network. Specifically, the image is first compressed, then the SVM network identifies the current orientation of the delivery receipt. If the current orientation is not oriented correctly, a prompt message is sent, followed by taking a picture of the delivery receipt and determining its orientation.

[0070] The verification element acquisition module includes a YOLO network verification element detection submodule, a key region cropping submodule, and a text geometric transformation submodule. The YOLO network verification element detection submodule uses YOLO's anchor boxes to simultaneously verify whether a signature or seal is present and the shape of the seal. The key region cropping submodule verifies the identified signature and seal according to a preset area, and cropps them after successful verification. The text geometric transformation submodule uses image processing technology, simulating a large number of customized machine-printed text images during the training phase for ORC recognition.

[0071] Text recognition (OCR) module: Transformer's end-to-end OCR recognition technology, which can recognize curved surfaces, handwritten and machine-printed text. If there is a new seal, during the training phase, it is often only necessary to annotate the text of the new seal, and then retrain it to recognize the new seal without having to retrain on the shape of the seal.

[0072] Information matching module: During the verification phase, we adopted a verification method based on CER (Character Error Rate) to give the system a certain degree of error tolerance.

[0073] Among them, machine learning includes neural networks, SVM (Support Vector Machine) for classification, OCR (Optical Character Recognition), etc. This receipt system is a system that integrates multiple machine learning algorithms. The loss function can be a calculation function of the difference between the predicted value and the actual value, and is a function for training the parameters of each node of the neural network. The loss function largely determines the quality of the neural network and is the core of the neural network. The neural network can be a linear fitting algorithm modeled after the neural network, including two parts: training and testing. During training, the coefficients of each neuron (node) are determined through the loss function, and during testing, the type and position of the object are deduced based on the coefficients. The confidence level can be the probability of the target belonging to a certain category. For example, if the probability that the target is a bird is 0.7, the probability that it is an airplane is 0.2, and the probability that it is a cat is 0.01, then it is said that the confidence level that the target is a bird is high, at 0.7.

[0074] The YOLO neural network, whose full English name is You Only Look Once, literally means just look once, meaning that when there are multiple targets, all targets can be found as long as one operation is performed. It is a very new neural network. For the receipt, it can find verification elements such as signatures, seals, dates, ID numbers, etc. at one time.

[0075] In this solution, YOLO uses anchor boxes, which are also called anchor frames or prior boxes. For verification element detection, a set of boxes are pre-drawn during the training of the neural network, and one of the boxes in this set is used as the prior box for the verification element to find the verification element and its position. This solution also involves the determination of the receptive field. When searching for verification elements, instead of directly searching for verification elements on the original image, the original image is scaled down, and the verification elements are found on the scaled-down image and then mapped back to the original image. The size of the original image represented by the scaled-down image is the receptive field. It is equivalent to the scale of a map.

[0076] OCR, whose full English name is Optical Character Recognition, can recognize printed characters, the circular characters of seals, and handwritten characters.

[0077] CER: The full English name is Character Error Rate. It verifies whether a certain character in a phrase is incorrect and the degree of error through the phrase. For example, when recognizing the handwritten word "wooden barrel" as "wooden bucket", it is considered that the actual handwritten word is "wooden barrel".

[0078] Based on determining the direction of the signed receipt, this solution, due to the relatively fixed format of the signed receipt, not only relies on the shape of the verification element during the detection of verification elements, but also combines the confidence level of the type of verification element with the location of the verification element to improve the prediction accuracy of the type of verification element. The CER method is used to give the system a certain fault tolerance, tolerating text recognition errors caused by overlapping text and stamps and paper creases in the signed receipt, thereby improving the accuracy of text recognition.

[0079] In this technical solution, optionally, after using the YOLO network to detect verification elements in the signed receipt image and determine the border position and category of the verification elements, the method further includes:

[0080] Identify whether the category of the verification element includes the category of verification elements identified by the signed receipt information;

[0081] If not, a message indicating that the signature information recognition failed will be generated.

[0082] Optionally, in this technical solution, a YOLO network is used to detect verification elements in the signed receipt image, determine the border position and category of the verification elements, and identify whether the border position and category of each verification element in the signed receipt image match, including:

[0083] The signed receipt image is input into the YOLO network to obtain the border position and category of the verification element in the signed receipt image;

[0084] The system identifies whether the border position and category of the verification element are within a preset verification element area. If the border position of the verification element is within the preset verification element area and the category of the verification element corresponds to the preset verification element area, then the verification element is identified as existing. If the border position of the verification element is not within the preset verification element area and the category of the verification element does not correspond to the preset verification element area, then the verification element is identified as not existing.

[0085] Understandably, after the delivery receipt is provided to the recipient, the recipient will sign for it, either by handwriting or by affixing a company seal. In this case, the main task of the YOLO network is to identify the recipient's information to facilitate subsequent verification tasks.

[0086] The advantage of this scheme is that YOLO verification element detection can calculate multiple targets at once, but it is unrelated to the location of the verification element. The positions of the signature, seal, and date on the receipt are relatively fixed. After detecting the verification element, if the verification element is outside the specified range, the detected verification element is considered as background. Thus, a simple algorithm is used to verify the relationship between element type and position.

[0087] In this technical solution, optionally, the text recognition model training phase includes:

[0088] Acquire a preset number of curved text images, machine-printed text images, and handwritten text images;

[0089] The curved text image is scaled, cropped, mirrored, and rotated proportionally; the machine-printed text image is scaled, cropped, and mirrored proportionally; and the handwritten text image is scaled, cropped, and mirrored proportionally to obtain sample data.

[0090] A character recognition model is trained based on the sample data.

[0091] The advantage of this approach is that by using text geometric transformations, a large number of customized curved surfaces and machine-printed text images are simulated during the training phase. When a new stamp or text is added, there is no need to retrain the stamp; only the new text needs to be recognized, thus reducing training time.

[0092] Figure 3 This is a schematic diagram illustrating the process of recognizing receipt information based on a terminal device, as provided in an embodiment of this application. For example... Figure 3 As shown, the process includes:

[0093] S310: The signed receipt scan is used as the original image. The current angle of the signed receipt image is calculated through the SVM network, and the signed receipt is rotated to the positive position according to the current angle.

[0094] S320: Use the YOLO network to calculate the borders of the detected verification elements on the original image and obtain the category of the verification elements. Check whether the position of the border matches the category of the verification element, whether it contains the required signature, whether there is a stamp, and the shape of the stamp.

[0095] S330, crop the area of ​​the verification element based on the border of the verification element on the original image;

[0096] During the training phase, a large number of curved text (stamp) and machine-printed text images were customized for the S340.

[0097] S350 inputs the curved text on the seal, signature, handwritten text, and machine-printed text into the text recognition module for text recognition;

[0098] S360 verifies whether the shape of the seal, the seal and the text on the seal, the signed ID number, and whether the products listed on the receipt are specific types of products.

[0099] The technical solution provided in this embodiment uses SVM to determine the direction of the receipt, which reduces the amount of computation required to determine the direction of the receipt compared to CNN. It uses a preset area for verification elements to detect the existence of necessary elements, which simplifies the amount of computation required for detection. Only curved text and machine-printed text are simulated and trained during training, so that when there is new stamp text, only the newly added text is trained. This simplifies the amount of computation in many ways, making it possible to detect receipts using mobile terminals.

[0100] In this solution, optionally, the border position of each verification element is calculated, and each verification element is cropped and categorized according to the border position, including:

[0101] The center point and border position of the verification element in the signed receipt image are calculated based on the receptive field.

[0102] Based on the center point and border position of the verification element in the receipt image, calculate the border position of each verification element, and then crop and classify and store them.

[0103] This scheme is designed to accurately determine the position of the cropped borders, and after cropping each verification element according to the border position, it is classified and stored, which can improve the efficiency of subsequent text content recognition.

[0104] Since the receipt is a scanned document, there may be instances of unclear scanning during the process. Additionally, overlapping of stamps, signatures, dates, etc., can also affect text recognition. Performing text similarity matching to filter out interfering pixels caused by scanning and element overlap can improve text recognition accuracy.

[0105] The technical solutions provided in the above embodiments improve the confidence of element categories by using preset element regions and improve the detection of overlapping elements. In addition, ECR and other methods can be used to filter out the interference to text caused by scanning and element overlap, thereby improving the accuracy of the receipt from multiple aspects.

[0106] Example 2

[0107] Figure 4 This is a schematic diagram of the terminal device-based signature information recognition device provided in Embodiment 2 of this application. Figure 4 As shown, the device includes:

[0108] Angle processing unit 410 is used to acquire a signed receipt image, identify the current angle of the signed receipt image through an SVM network, and generate a rotation command to rotate the signed receipt image to a standard angle if the current angle is not a standard angle.

[0109] The verification element detection unit 420 is used to perform verification element detection in the signed receipt image using the YOLO network, determine the border position and category of the verification element, and identify whether the border position and category of each verification element in the signed receipt image match.

[0110] The verification element clipping unit 430 is used to clip each verification element according to the border position if a match is found.

[0111] The text recognition unit 440 is used to input the cropped verification elements into the text recognition model for text recognition.

[0112] The information recognition result determination module 450 is used to determine that the information recognition of the receipt is complete if the text recognition result matches the pre-registered receipt information.

[0113] Furthermore, the device also includes an element category matching verification unit, used for:

[0114] Identify whether the category of the verification element includes the category of verification elements identified by the signed receipt information;

[0115] If not, a message indicating that the signature information recognition failed will be generated.

[0116] Furthermore, the angle processing unit 410 includes:

[0117] The normalization processing subunit is used to scan the receipt to obtain the original image of the receipt, perform normalization processing on the original image of the receipt, and convert the original image of the receipt into a uniform and fixed height and width as the receipt image.

[0118] An angle recognition subunit is used to downsample the pixels of the signed receipt image and input them into the SVM network to determine whether the current angle of the signed receipt is positive.

[0119] The rescan prompt sub-unit is used to prompt a rescan of the receipt when the receipt is not in the forward direction.

[0120] Furthermore, the verification element detection unit 420 includes:

[0121] The verification element identification subunit is used to input the signed receipt image into the YOLO network to obtain the border position and category of the verification element in the signed receipt image.

[0122] The matching verification subunit is used to identify whether the border position and category of the verification element are within the preset verification element area. If the border position of the verification element is within the preset verification element area and the category of the verification element corresponds to the preset verification element area, then the verification element is identified as existing. If the border position of the verification element is not within the preset verification element area and the category of the verification element does not correspond to the preset verification element area, then the verification element is identified as not existing.

[0123] Furthermore, the device also includes a model training unit, which includes:

[0124] The image acquisition subunit is used to acquire a preset number of curved text images, machine-printed text images, and handwritten text images;

[0125] The sample production subunit is used to scale, crop, mirror, and rotate the curved text image proportionally, scale, crop, and mirror the machine-printed text image proportionally, and scale, crop, and mirror the handwritten text image proportionally to obtain sample data.

[0126] The model training subunit is used to train a text recognition model based on the sample data.

[0127] Furthermore, the device also includes:

[0128] The model storage unit is used to store the text recognition model in a preset storage location of the terminal device, so as to call the text recognition model after cropping each verification element according to the border position.

[0129] This device can execute the terminal device-based receipt information recognition method provided in the above embodiments, and has corresponding functional units and beneficial effects. Further details are omitted here.

[0130] Example 3

[0131] This application embodiment also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a method for identifying receipt information based on a terminal device. The method includes:

[0132] The system acquires a signed receipt image, identifies the current angle of the signed receipt image using an SVM network, and generates a rotation command to rotate the signed receipt image to a standard angle if the current angle is not a standard angle.

[0133] The YOLO network is used to detect verification elements in the signed receipt image, determine the border position and category of the verification elements, and identify whether the border position and category of each verification element in the signed receipt image match.

[0134] If a match is found, each verification element is cropped according to the border position.

[0135] The cropped verification elements are input into the text recognition model for text recognition.

[0136] If the text recognition result matches the pre-registered receipt information, then the information recognition of the receipt is considered complete.

[0137] Storage medium – any type of memory electronic device or storage electronic device. The term “storage medium” is intended to include: mounting media, such as CD-ROMs, floppy disks, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disks or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. Furthermore, storage media may reside in a computer system in which a program is executed, or may reside in a different second computer system connected to the computer system via a network (such as the Internet). The second computer system can provide program instructions to the computer for execution. The term “storage medium” can include two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). Storage media may store program instructions (e.g., specifically implemented as a computer program) that can be executed by one or more processors.

[0138] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the terminal device-based receipt information identification operation as described above, but can also execute related operations in the terminal device-based receipt information identification method provided in any embodiment of this application.

[0139] Example 4

[0140] This application provides an electronic device. Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application. Figure 5 As shown, this embodiment provides an electronic device 500, which includes: one or more processors 520; and a storage device 510 for storing one or more programs. When the one or more programs are run by the one or more processors 520, the one or more processors 520 implement the terminal device-based signature information identification method provided in this application embodiment. The method includes:

[0141] The system acquires a signed receipt image, identifies the current angle of the signed receipt image using an SVM network, and generates a rotation command to rotate the signed receipt image to a standard angle if the current angle is not a standard angle.

[0142] The YOLO network is used to detect verification elements in the signed receipt image, determine the border position and category of the verification elements, and identify whether the border position and category of each verification element in the signed receipt image match.

[0143] If a match is found, each verification element is cropped according to the border position.

[0144] The cropped verification elements are input into the text recognition model for text recognition.

[0145] If the text recognition result matches the pre-registered receipt information, then the information recognition of the receipt is considered complete.

[0146] Figure 5 The electronic device 500 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0147] like Figure 5 As shown, the electronic device 500 includes a processor 520, a storage device 510, an input device 530, and an output device 540; the number of processors 520 in the electronic device can be one or more. Figure 5 Taking a processor 520 as an example; the processor 520, storage device 510, input device 530, and output device 540 in the electronic device can be connected via a bus or other means. Figure 5 Taking the connection via bus 550 as an example.

[0148] The storage device 510, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and module units, such as the program instructions corresponding to the terminal device-based signature information identification method in the embodiments of this application.

[0149] Storage device 510 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on terminal usage. Furthermore, storage device 510 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, storage device 510 may further include memory remotely located relative to processor 520, and these remote memories can be connected via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0150] Input device 530 can be used to receive input digital, character, or voice information, and to generate key signal inputs related to user settings and function control of the electronic device. Output device 540 may include electronic devices such as a display screen and a speaker.

[0151] The electronic device provided in this application simplifies the computation of determining the direction of the receipt by replacing the CNN with an SVM, thus transforming the original second-order recognition network into a first-order one. Furthermore, since the format of the receipt is relatively fixed, after detecting the category of the verification element, the category to which the verification element belongs can be quickly determined based on its location, simplifying the calculation of the confidence score of the verification element category. Moreover, during the training phase, a large number of customized seals and machine-printed text are introduced, greatly improving the scalability of the model. When a new seal is added, only the text of the new seal needs to be trained, shortening the training time.

[0152] The terminal device-based receipt information recognition device, medium, and equipment provided in the above embodiments can run the terminal device-based receipt information recognition method provided in any embodiment of this application, and have the corresponding functional modules and beneficial effects of running the method. Technical details not described in detail in the above embodiments can be found in the terminal device-based receipt information recognition method provided in any embodiment of this application.

[0153] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0158] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0159] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0160] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0161] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for identifying receipt information based on terminal devices, characterized in that, The method is executed by a terminal device, and the method includes: The system acquires a signed receipt image, identifies the current angle of the signed receipt image using an SVM network, and generates a rotation command to rotate the signed receipt image to a standard angle if the current angle is not a standard angle. The signed receipt image is input into the YOLO network to obtain the border position and category of the verification element in the signed receipt image; The system identifies whether the border position and category of the verification element are within a preset verification element area. If the border position of the verification element is within the preset verification element area and the category of the verification element corresponds to the preset verification element area, then the verification element is identified as existing. If the border position of the verification element is not within the preset verification element area and the category of the verification element does not correspond to the preset verification element area, then the verification element is identified as not existing. The verification elements include the stamp, customer's handwritten signature, order number, and date in the receipt image. If a match is found, each verification element is cropped according to the border position. The cropped verification elements are input into the text recognition model for text recognition. Specifically, the key area cropping submodule of the verification element acquisition module is used to recognize curved text, handwritten text, and machine-printed text. Curved text corresponds to the text contained in the stamp when the enterprise signes for the goods. Handwritten text corresponds to the name of the signing unit or person handwritten during the signing process. Handwritten text is classified and recognized according to different fonts. Machine-printed text corresponds to the text included in the machine-printed information contained in the signing form. If the text recognition result matches the pre-registered receipt information, then the information recognition of the receipt is considered complete. After using the YOLO network to detect verification elements in the signed receipt image and determine the border position and category of the verification elements, the method further includes: Identify whether the category of the verification element includes the category of verification elements identified by the signed receipt information; If not, a message indicating that the signature information recognition failed will be generated.

2. The method according to claim 1, characterized in that, Acquire a signed receipt image, identify the current angle of the signed receipt image using an SVM network, and if the current angle is not a standard angle, generate a rotation command to rotate the signed receipt image to a standard angle, including: Scan the receipt to obtain the original image of the receipt, normalize the original image of the receipt, and convert the original image of the receipt into a uniform and fixed height and width as the receipt image; The pixels of the signed receipt image are downsampled and input into the SVM network to determine whether the current angle of the signed receipt is positive. If the receipt is not in the positive direction, prompt the recipient to rescan the receipt.

3. The method according to claim 1, characterized in that, The text recognition model training phase includes: Acquire a preset number of curved text images, machine-printed text images, and handwritten text images; The curved text image is scaled, cropped, mirrored, and rotated proportionally; the machine-printed text image is scaled, cropped, and mirrored proportionally; and the handwritten text image is scaled, cropped, and mirrored proportionally to obtain sample data. A character recognition model is trained based on the sample data.

4. The method according to claim 3, characterized in that, The method further includes: The text recognition model is stored in a preset storage location on the terminal device, so that after each verification element is cropped according to the border position, the text recognition model can be called.

5. A receipt information recognition device based on a terminal device, characterized in that, The device is configured in a terminal device, and the device includes: An angle processing unit is used to acquire a signed receipt image, identify the current angle of the signed receipt image through an SVM network, and generate a rotation command to rotate the signed receipt image to a standard angle if the current angle is not a standard angle. The verification element detection unit is used to input the signed receipt image into the YOLO network to obtain the border position and category of the verification element in the signed receipt image; identify whether the border position and category of the verification element are within a preset verification element area. If the border position of the verification element is within the preset verification element area and the category of the verification element corresponds to the preset verification element area, then the verification element is identified as existing; if the border position of the verification element is not within the preset verification element area and the category of the verification element does not correspond to the preset verification element area, then the verification element is identified as not existing. The verification elements include a stamp, customer's handwritten signature, order number, and date in the signed receipt image. The verification element cropping unit is used to crop each verification element according to the border position if a match is found. The text recognition unit is used to input the cropped verification elements into the text recognition model for text recognition. Specifically, it uses OCR to recognize the curved text, handwritten text, and machine-printed text obtained by the key area cropping submodule of the verification element acquisition module. The curved text corresponds to the text contained in the stamp when the enterprise signes for the goods. The handwritten text corresponds to the name of the signing unit or person handwritten during the signing process. The handwritten text is classified and recognized according to different fonts. The machine-printed text corresponds to the text included in the machine-printed information contained in the signing form. The information recognition result determination module is used to determine that the information recognition of the receipt is complete if the text recognition result matches the pre-registered receipt information. After using the YOLO network to detect verification elements in the signed receipt image and determine the border position and category of the verification elements, the method further includes: Identify whether the category of the verification element includes the category of verification elements identified by the signed receipt information; If not, a message indicating that the signature information recognition failed will be generated.

6. The apparatus according to claim 5, characterized in that, The device further includes a verification element category matching unit, used for: Identify whether the category of the verification element includes the category of verification elements identified by the signed receipt information; If not, a message indicating that the signature information recognition failed will be generated.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the terminal device-based signature information recognition method as described in any one of claims 1-4.

8. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the terminal device-based signature information recognition method as described in any one of claims 1-4.

Images