Bill damage detection method, device, equipment, medium and program product

Abstract

The application discloses a kind of bill residual damage detection method, device, equipment, medium and program product, it is related to artificial intelligence technical field.The method comprises: using first target detection model to determine the target bill type of original bill image and bill region of interest image, and according to target bill type, extract bill key information from bill region of interest image;Based on bill key information, using second target detection model to carry out residual damage detection to bill region of interest image, obtain initial residual damage detection result;Initial residual damage detection result is processed based on bill key information, and target residual damage detection result is obtained;Target residual damage detection result includes target residual damage type and target residual damage area.The technical scheme of the embodiment of the application eliminates background interference, improves the real-time performance of residual damage bill detection, strengthens the feature extraction of slight residual damage, solves the problem that slight residual damage feature is weak and easy to be ignored, and improves the accuracy of bill slight residual damage detection.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, equipment, medium, and program product for detecting damaged documents. Background Technology

[0002] With the deepening of digital transformation in business operations, the electronic imaging processing of paper documents has become a core element in improving business efficiency and reducing operating costs. However, documents are prone to damage during circulation, collection, and storage. Among these, minor damage accounts for more than 60%. Due to their subtle characteristics, such damage is easily overlooked by traditional detection models. Furthermore, document collection scenarios are complex, often accompanied by interference from complex backgrounds such as desktops, wallets, and coins. Moreover, noise from scanner glass dust and light reflection further exacerbates the difficulty of detection.

[0003] Existing invoice detection technologies have significant shortcomings: First, they cannot cover the unique minor damage types of invoices, resulting in a missed detection rate of over 20% for difficult samples; second, they suffer from severe background interference, leading to significant waste of computing power, with the detection time for a single invoice exceeding 500ms, which cannot meet the real-time requirements of edge devices. Summary of the Invention

[0004] This invention provides a method, apparatus, equipment, medium, and program product for detecting damaged documents, which eliminates background interference and improves the real-time performance of damaged document detection; at the same time, it enhances the feature extraction of minor damage, solves the problem that the features of minor damage are weak and easily ignored, and improves the accuracy of minor damage detection of documents.

[0005] According to one aspect of the present invention, a method for detecting damaged documents is provided, the method comprising: The first object detection model is used to determine the target bill type and the region of interest (ROI) image of the original bill image, and key bill information is extracted from the ROI image based on the target bill type. Based on the key information of the invoice, a second target detection model is used to perform damage detection on the region of interest image of the invoice to obtain an initial damage detection result. The initial damage detection result is processed based on the key information of the ticket to obtain the target damage detection result; the target damage detection result includes the target damage type and the target damage area.

[0006] According to another aspect of the present invention, a document damage detection device is provided, the device comprising: The bill key information detection module is used to determine the target bill type and the region of interest (ROI) image of the original bill image using a first target detection model, and to extract key bill information from the ROI image based on the target bill type. The initial damage detection module is used to perform damage detection on the region of interest image of the ticket based on the key information of the ticket and using the second target detection model to obtain the initial damage detection result; The target damage detection module is used to process the initial damage detection result based on the key information of the ticket to obtain the target damage detection result; the target damage detection result includes the target damage type and the target damage area.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the document damage detection method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the document damage detection method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the document damage detection method according to any embodiment of the present invention.

[0010] The technical solution of this invention employs a first target detection model to determine the target ticket type and the region of interest (ROI) image of the original ticket image. Based on the target ticket type, key ticket information is extracted from the ROI image. Firstly, the first target detection model clarifies the range of the ROI image, eliminating information interference from ticket damage detection, reducing computational power consumption, and improving the real-time performance and accuracy of ticket damage detection. Simultaneously, it extracts key ticket information as a reference, providing data support for subsequent damage detection. Furthermore, based on the key ticket information, a second target detection model is used to perform damage detection on the ROI image, obtaining an initial damage detection result. This initial damage detection result is then processed based on the key ticket information to obtain the target damage detection result. By fusing the key ticket information through a dual-target detection model, the feature extraction of minor damage is enhanced, solving the problem of weak and easily overlooked features of minor damage. This significantly improves the recognition rate of minor ticket damage such as minor creases, light-colored stains, and edge damage, effectively reducing missed detections and improving the accuracy of minor ticket damage detection.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a document damage detection method according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of a document damage detection method according to Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of a damaged document detection device according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the document damage detection method according to an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Example 1 Figure 1This is a flowchart illustrating a method for detecting document damage according to Embodiment 1 of the present invention. This embodiment of the invention is applicable to detecting documents with different types of damage. The method can be executed by a document damage detection device, which can be implemented in hardware and / or software. This device can be configured in an electronic device that performs document damage detection, such as a client or server.

[0017] See Figure 1 The document damage detection method shown includes: S101. The first target detection model is used to determine the target ticket type and the region of interest (ROI) image of the original ticket image, and the key information of the ticket is extracted from the ROI image based on the target ticket type.

[0018] The original ticket image is the original image of the ticket to be damaged. This original ticket image may contain background and clutter. The first object detection model is used to identify the target ticket type and the region of interest (ROI) image of the ticket within the original ticket image. The input data for the first object detection model is the original ticket image; the output is the target ticket type and the ROI image of the original ticket image. In essence, the first object detection model performs two core tasks: first, accurately determining the "target ticket type" of the ticket contained in the original ticket image; and second, selecting the "ROI image" from the original ticket image.

[0019] Optionally, the first object detection model is pre-trained using preset training parameters and a document feature library. Optionally, since there are few slightly damaged training samples, to increase the number of slightly damaged samples, data augmentation operations such as perspective transformation, elastic transformation, and complex background synthesis can be performed on the sample documents in the document feature library. This can enhance the robustness of the first object detection model to complex acquisition environments and improve its anti-interference ability. For example, perspective transformation can correct the tilt angle of the sample document by 0-20°. Elastic transformation can be used to bend and deform the sample document. Complex background synthesis can be used to attach the sample document to background images such as desktops, wallets, and piles of documents, with the background accounting for 30%-70% of the document, thus synthesizing the sample document.

[0020] The target invoice type characterizes the specific type of invoice contained in the original invoice image. For example, target invoice types include value-added tax invoices, bank acceptance drafts, or train tickets. Different target invoice types correspond to different key invoice information. The texture features corresponding to different target invoice types also differ.

[0021] The region of interest (ROI) image of a ticket is used to characterize the area containing the main body of the ticket within the original ticket image. By filtering the ROI image from the original ticket image, irrelevant interference such as background, shooting shadows, and clutter can be eliminated, focusing on the image information of the ticket itself.

[0022] Key information on a negotiable instrument refers to the core functional information of the instrument contained in the original image of the instrument. Optionally, key information can be structured text information. For example, key information may include the instrument number, date of issue, amount, account number, and seal identification.

[0023] Specifically, the original invoice image is input into the first object detection model, which outputs the target invoice type and the region of interest (ROI) image. Based on the target invoice type, the location of key information corresponding to the invoice in the ROI image is determined, and information is extracted from the location of the key information to obtain the key information of the invoice.

[0024] S102. Based on the key information of the bill, the second target detection model is used to perform damage detection on the region of interest image of the bill to obtain the initial damage detection result.

[0025] The second object detection model is used to perform initial damage detection on the region of interest (ROI) image of the ticket. The input data for the second object detection model are the ROI image of the ticket and key information about the ticket; the output is the initial damage detection result. The initial damage detection result characterizes the preliminary detection result of damage to the ticket contained in the ROI image. For example, the initial damage detection result includes the initial damage type, the initial damage region, and the confidence level. Specifically, the initial damage type characterizes the preliminary detection result of the damage type of the ticket contained in the ROI image. For example, the initial damage type characterizes the degree of damage to the ticket contained in the ROI image. The initial damage region is the preliminary detection result of the damaged region of the ticket contained in the ROI image. The confidence level characterizes the reliability of the initial damage detection result.

[0026] Specifically, the key information of the bill and the image of the region of interest of the bill are input into the second target detection model, and the initial damage detection results are output.

[0027] In an optional embodiment of the present invention, based on key information of the invoice, a second target detection model is used to perform damage detection on the region of interest image of the invoice to obtain an initial damage detection result. This includes: inputting the region of interest image of the invoice into the main branch of the second target detection model, and inputting the key information of the invoice into the context branch of the second target detection model to extract features respectively, thereby obtaining the main features of the invoice and key context features; fusing the main features of the invoice and key context features to obtain fused features; predicting candidate boxes for damage based on the fused features, and filtering the candidate boxes for damage to obtain the initial damage detection result.

[0028] The backbone branch is used to extract visual features from the region of interest (ROI) image of the ticket. For example, the backbone branch may contain convolutional layers and pooling layers. The context branch is used to extract features from key information on the ticket. The main features of the ticket are the visual features of the ticket contained within the ROI image. The main features primarily capture damage-related signals on the ticket surface and are the fundamental features for identifying surface damage. For example, the main features include the texture, grayscale variations, and edge contours of the ticket surface. Key context features are used to highlight the feature weights of the regions containing key information on the ticket. For example, key context features include the feature vectors of the key information on the ticket. By introducing key context features, the second object detection model will focus on minor damage in the regions containing key information on the ticket during subsequent detection processes, such as fine creases or light-colored stains in the ticket number area, thus solving the problem that minor damage features are not obvious and are easily overlooked.

[0029] The fusion feature is the result of fusing the main feature and key contextual features. The fusion feature retains the overall visual information of the ticket image contained within the region of interest (ROI) image, while also incorporating contextual guidance from key information about the ticket, effectively enhancing the feature signals of minor damage. For example, previously blurred fine creases become more prominent under the guidance of key contextual features. Damage candidate boxes are used to identify areas on the ticket within the ROI image that may contain damage. One damage candidate box corresponds to one suspected damaged area. For example, the damage candidate box contains preliminary information such as the location, damage type, and confidence level of the suspected damaged area.

[0030] Specifically, the region of interest (ROI) image of the ticket is input into the main branch of the second object detection model for feature extraction, outputting the ticket's main features. Key information about the ticket is input into the context branch of the second object detection model for feature extraction, obtaining key context features. Channel concatenation and attention enhancement are used to fuse the ticket's main features and key context features, resulting in fused features. The second object detection model predicts damaged candidate boxes based on these fused features. The confidence scores of each damaged candidate box are compared to a pre-set confidence score. Damaged candidate boxes with confidence scores greater than or equal to the pre-set confidence score are identified as initial damaged detection results. The pre-set confidence score is a pre-defined lower limit for initially identifying damaged candidate boxes. This pre-set confidence score is used for initial screening of damaged candidate boxes.

[0031] This scheme introduces a dual-branch feature extraction method. The main branch and context branch of the second target detection model are used to extract features from the region of interest (ROI) image and key information of the ticket, respectively, yielding the main features and key context features. This provides data support for subsequent damage identification and improves the recognition rate of minor damage. By fusing the main features and key context features, two types of features are integrated. Based on the fused features, candidate damage boxes are predicted, achieving preliminary localization of suspected damaged areas. Filtering the candidate damage boxes yields initial damage detection results, enabling preliminary classification of damage types. In the process of ticket damage detection, both the overall visual information of the ticket image and the contextual guidance of key information are considered, effectively strengthening the feature signals of minor damage and further improving the accuracy of minor ticket damage detection.

[0032] In an optional embodiment of the present invention, filtering the damaged candidate boxes includes: determining an initial damage type based on the area of ​​the damaged candidate box; the initial damage type being either small target damage or non-small target damage; if the initial damage type is small target damage, then filtering the damaged candidate boxes using a first intersection-union threshold; if the initial damage type is non-small target damage, then filtering the damaged candidate boxes using a second intersection-union threshold.

[0033] The initial damage type is used to preliminarily characterize the degree of damage to the document corresponding to the damage candidate frame. The degree of damage for small-target damage is less than that for non-small-target damage. This can be understood as the area of ​​the damage candidate frame corresponding to small-target damage being smaller than the area of ​​the damage candidate frame corresponding to non-small-target damage. Small-target damage refers to relatively minor document damage. For example, small-target damage includes minor creases, light-colored stains, and minor tears. Minor creases are thin creases with a width between 0.3mm and 1mm. Light-colored stains are stains with a grayscale difference between 10 and 30. Minor tears are tears with a damaged area less than 5mm². 2The damaged area is defined as the area of ​​damage to a small target. Non-small target damage refers to any damage other than small target damage. Comparatively, small target damage has weaker characteristics and is more easily missed. Non-small target damage has relatively obvious characteristics and is easier to identify.

[0034] The Cross-Union Ratio (CIRR) threshold is used to measure the degree of overlap between the candidate bounding boxes of damage predicted by the second object detection model and the actual damaged areas. A higher CIRR threshold indicates a higher degree of overlap between the candidate bounding boxes and the actual damaged areas, resulting in higher accuracy in damage detection; conversely, a lower CIRR threshold indicates a lower degree of overlap, resulting in lower accuracy in damage detection. The first CIRR threshold is used to filter candidate bounding boxes corresponding to small target damage. For example, the first CIRR threshold can be 0.3-0.4. The second CIRR threshold is used to filter candidate bounding boxes corresponding to non-small target damage. For example, the second CIRR threshold can be 0.5.

[0035] The first cross-union ratio (CUN) threshold is lower than the second CUN threshold. This can be understood as follows: because small target defects are small in size and have low overlap, using a higher CUN threshold can easily lead to false negatives due to the low overlap between candidate defect boxes and the actual defect area. By using a smaller first CUN threshold to filter candidate defect boxes for small target defects, it ensures that small target defects are not missed. On the other hand, non-small target defects are larger in size and have relatively obvious features. To ensure detection accuracy and reduce false positives, using a larger second CUN threshold to filter candidate defect boxes for non-small target defects can identify candidate defect boxes with higher overlap and closer resemblance to the actual defect area. By explicitly specifying that the first CUN threshold is lower than the second CUN threshold, the problem of missed detection for small target defects and the detection accuracy for non-small target defects can be balanced, improving the recognition rate of minor defects.

[0036] Specifically, the area of ​​the damaged candidate box is compared with a preset area of ​​slight damage. If the area of ​​the damaged candidate box is less than or equal to the preset area of ​​slight damage, the initial damage type is determined to be small target damage; if the area of ​​the damaged candidate box is less than or equal to the preset area of ​​slight damage, the initial damage type is determined to be non-small target damage. If the initial damage type is small target damage, the intersection-union ratio (IU / R) of the second target detection model is adjusted to the first IU / R threshold to filter the damaged candidate boxes. If the initial damage type is non-small target damage, the IU / R of the second target detection model is adjusted to the second IU / R threshold to filter the damaged candidate boxes.

[0037] This scheme determines the initial damage type by the area of ​​the candidate damage bounding box and differentiates the cross-union threshold (CUI) for different initial damage types. Considering that small target damage is small in size and easily misidentified due to low overlap between the candidate damage bounding box and the actual damage area, resulting in missed detections, a smaller first CUI threshold is used to screen candidate damage bounding boxes for small target damage, which helps improve the sample recall rate of small target damage and ensures that small target damage is not missed. On the other hand, non-small target damage is larger in size and has relatively obvious features. To ensure detection accuracy and reduce false detections, a larger second CUI threshold is used to screen candidate damage bounding boxes for non-small target damage, which can screen out candidate damage bounding boxes with higher overlap and closer to the actual damage area, thereby improving the detection accuracy of non-small target damage. By specifying that the first CUI threshold is smaller than the second CUI threshold, the problem of missed detection of small target damage and the detection accuracy of non-small target damage can be taken into account, which can improve the recognition rate of minor damage.

[0038] S103. Process the initial damage detection results based on the key information of the invoice to obtain the target damage detection results.

[0039] In the original image of a ticket, the key information of the ticket is more important than other information on the ticket. Therefore, a more precise damage verification method is needed to ensure the accuracy of damage identification for the key information. The target damage detection result is used to characterize the final detection result of damage detection for the tickets contained in the region of interest image. For example, the target damage detection result includes the target damage type and the target damage region. The target damage type is used to characterize the final detection result of the damage type of the tickets contained in the region of interest image. For example, the target damage type characterizes the degree of damage to the tickets contained in the region of interest image. The target damage region is the final detection result of the damaged region of the tickets contained in the region of interest image.

[0040] Specifically, it is determined whether the initial damage detection result is associated with the key information of the invoice. If the initial damage detection result is associated with the key information of the invoice, then an edge detection operator is used to extract the information edges of the initial damaged area contained in the initial damage detection result, and edge continuity detection and contour closure detection are performed on the extracted information. If the extracted information edges are continuous and the contour is closed, the initial damage detection result is discarded; if the extracted information edges are broken, missing, or the contour is not closed, the initial damage detection result is retained and determined as the target damage detection result. If the initial damage detection result is associated with the key information of the invoice, then the initial damage detection result is determined as the target damage detection result.

[0041] The technical solution of this invention employs a first target detection model to determine the target ticket type and the region of interest (ROI) image of the original ticket image. Based on the target ticket type, key ticket information is extracted from the ROI image. Firstly, the first target detection model clarifies the range of the ROI image, eliminating information interference from ticket damage detection, reducing computational power consumption, and improving the real-time performance and accuracy of ticket damage detection. Simultaneously, it extracts key ticket information as a reference, providing data support for subsequent damage detection. Furthermore, based on the key ticket information, a second target detection model is used to perform damage detection on the ROI image, obtaining an initial damage detection result. This initial damage detection result is then processed based on the key ticket information to obtain the target damage detection result. By fusing the key ticket information through a dual-target detection model, the feature extraction of minor damage is enhanced, solving the problem of weak and easily overlooked features of minor damage. This significantly improves the recognition rate of minor ticket damage such as minor creases, light-colored stains, and edge damage, effectively reducing missed detections and improving the accuracy of minor ticket damage detection.

[0042] Example 2 Figure 2 This is a flowchart of a document damage detection method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment of the present invention specifies "processing the initial damage detection result based on key document information" as follows: "Determining whether the initial damaged area in the initial damage detection result is located within the key document area where the key document information is located; if the initial damaged area is located within the key document area, then performing grayscale difference amplification processing on the initial damaged area based on the standard lossless texture template of the key document area, and filtering the initial damage detection result according to the amplified grayscale difference result; if the initial damaged area is located outside the key document area, then calculating the texture similarity between the initial damaged area and the key document area, and filtering the initial damage detection result according to the texture similarity." Grayscale difference amplification filtering is used for the key document area to highlight minor damage features, improving the accuracy of minor damaged document detection; for non-key document areas, texture similarity filtering is used to eliminate normal texture interference, reducing missed detections and false detections, thus doubly ensuring the accuracy of document damage detection. It should be noted that parts not detailed in this embodiment of the present invention can be referred to in the descriptions of other embodiments.

[0043] See Figure 2 The document damage detection method shown includes: S201. The first target detection model is used to determine the target bill type and the region of interest (ROI) image of the original bill image, and the key information of the bill is extracted from the ROI image based on the target bill type.

[0044] S202. Based on the key information of the bill, the second target detection model is used to perform damage detection on the region of interest image of the bill to obtain the initial damage detection result.

[0045] S203. Determine whether the initial damaged area in the initial damage detection results is located within the key area of ​​the bill where the key information of the bill is located.

[0046] The critical area of ​​a negotiable instrument refers to the specific pixel area on the instrument where key information is contained. For example, the critical area includes the instrument number area, the amount area, and the seal / identification area. Since the critical area contains crucial information, even minor damage to this area significantly impacts the instrument's validity; therefore, its detection accuracy needs to be higher than that of other areas.

[0047] Specifically, the location range of the initial damaged area in the initial damage detection result is compared with the location range of the key area of ​​the bill where the key information is located. If the location range of the initial damaged area overlaps with the location range of the key area, then the initial damaged area in the initial damage detection result is determined to be located within the key area of ​​the bill where the key information is located. If the location range of the initial damaged area does not overlap with the location range of the key area, then the initial damaged area in the initial damage detection result is determined to be located outside the key area of ​​the bill where the key information is located.

[0048] S204. If the initial damaged area is located within the critical area of ​​the bill, the initial damaged area is subjected to grayscale difference amplification processing based on the standard lossless texture template of the critical area of ​​the bill, and the initial damaged detection result is filtered according to the amplified grayscale difference result to obtain the target damaged detection result.

[0049] Standard lossless texture templates serve as the core reference for normal textures. They provide lossless and clearly textured standard templates for key areas of various types of documents. Each standard lossless texture template corresponds to the target document type (or key area of ​​the document). The size, texture, and grayscale distribution of the standard lossless texture template perfectly match the key areas of the document.

[0050] The grayscale difference result is used to characterize the difference in grayscale values ​​between the initial damaged area and the standard textureless template. The magnified grayscale difference result is a magnified version of the grayscale difference result between the initial damaged area and the standard textureless template. Since the grayscale difference of minor damage (such as light stains and fine creases) is small and the damage features are not obvious, magnifying the grayscale difference result can make the originally inconspicuous damage features clear and distinguishable.

[0051] Specifically, if the initial damaged area is located within the critical area of ​​the document, the difference between the grayscale value of the initial damaged area and the grayscale value of the corresponding area in the standard lossless texture template of the critical area is calculated to obtain the grayscale difference value of the initial damaged area. Grayscale enhancement is then applied to the grayscale difference value of the initial damaged area to obtain the magnified grayscale difference value. This magnified grayscale difference value is then compared with a preset grayscale difference value. If the magnified grayscale difference value is less than the preset grayscale difference value, the initial damage detection result is filtered; if the magnified grayscale difference value is greater than or equal to the preset grayscale difference value, the initial damage detection result is retained, and the target damage detection result is obtained. The preset grayscale difference value is a pre-defined lower limit for the grayscale difference value when damage exists. The preset grayscale difference value is used to verify the initial damage detection result.

[0052] In an optional embodiment of the present invention, the initial damaged area is subjected to grayscale difference amplification processing based on a standard lossless texture template of the key area of ​​the bill, and the initial damaged detection result is filtered according to the amplified grayscale difference result, including: performing grayscale difference amplification processing on the initial damaged area based on a standard lossless texture template of the key area of ​​the bill to obtain the amplified grayscale difference result; if the amplified grayscale difference result reaches the corresponding damage determination threshold of the key area of ​​the bill, the initial damaged area is determined to be a real damage, and the corresponding initial damaged detection result is retained; otherwise, the initial damaged area is determined to be non-damaged, and the corresponding initial damaged detection result is discarded.

[0053] The damage assessment threshold is the lower limit of the grayscale difference when damage exists in the initial damaged area. This threshold is used to verify the initial damage detection results. The damage assessment threshold corresponds to the key areas of the document. This means that different types of key areas on a document can correspond to different damage assessment thresholds. Therefore, the damage assessment threshold can be dynamically adjusted based on the type of key area on the document, enabling differentiated verification of the initial damage detection results. For example, the damage assessment threshold for the document number area can be slightly lower to avoid missing subtle creases in the document number area. Conversely, the damage assessment threshold for the seal / stamp area can be slightly higher to reduce false detections caused by texture fluctuations.

[0054] True damage indicates that the initial damaged area has been verified to have damage. Non-damage indicates that the initial damaged area has been verified to not have damage. Non-damage may be caused by image noise, slight texture fluctuations, or shooting lighting deviations, etc.

[0055] Specifically, the difference between the grayscale value of the initial damaged area and the grayscale value corresponding to the standard lossless texture template of the key area of ​​the document is calculated to obtain the grayscale difference value of the initial damaged area. Grayscale enhancement is then applied to the grayscale difference value of the initial damaged area to obtain the magnified grayscale difference value. If the magnified grayscale difference value of the initial damaged area reaches the damage determination threshold of the corresponding key area of ​​the document, the initial damaged area is determined to be truly damaged, and the corresponding initial damage detection result is retained; otherwise, the initial damaged area is determined to be non-damaged, and the corresponding initial damage detection result is discarded.

[0056] This solution introduces a damage assessment threshold corresponding to key areas of the bill, dynamically adjusting the damage assessment threshold for different key areas of the bill, thereby further improving the accuracy of initial damage detection result screening. Simultaneously, if the magnified grayscale difference reaches the damage assessment threshold for the corresponding key area of ​​the bill, the initial damaged area is determined to be genuinely damaged, and the corresponding initial damage detection result is retained; otherwise, the initial damaged area is determined to be non-damaged, and the corresponding initial damage detection result is discarded. This ensures that genuine damage is not missed and non-damaged items are not misjudged, reducing missed and false detections, and further improving the screening efficiency and detection accuracy of initial damage detection results.

[0057] In an optional embodiment of the present invention, the grayscale difference of the initial damaged area is magnified based on the standard lossless texture template of the key area of ​​the bill to obtain the magnified grayscale difference result, including: calculating the grayscale difference between the standard lossless texture template of the key area of ​​the bill and the pixels in the initial damaged area to generate a grayscale difference map; and magnifying the grayscale difference map to obtain the magnified grayscale difference result.

[0058] The grayscale difference is the difference between the grayscale value of a single pixel in the initial damaged area and the grayscale value of the corresponding pixel in the standard textureless template. The grayscale value of a single pixel reflects the brightness of that location in the initial damaged area. The grayscale difference map is composed of the grayscale differences of all pixels in the initial damaged area. The grayscale difference map can visually display the distribution of the grayscale differences of each pixel in the initial damaged area. In the grayscale difference map, the larger the grayscale difference, the higher the probability that the pixel may be damaged.

[0059] Specifically, the difference between the grayscale values ​​of pixels in the initial damaged area and the corresponding grayscale values ​​of pixels in the standard lossless texture template of the key area of ​​the document is calculated to obtain the grayscale difference values ​​of each pixel in the initial damaged area, generating a grayscale difference map. The grayscale difference map is then linearly stretched or grayscale enhanced to obtain the magnified grayscale difference result.

[0060] This solution generates a grayscale difference map by using a standard non-destructive texture template of the key area of ​​the invoice and the grayscale difference of pixels in the initial damaged area. This allows for a more intuitive display of the distribution of grayscale differences among pixels in the initial damaged area. By magnifying the grayscale difference map, the magnified grayscale difference result is obtained, improving the convenience and efficiency of grayscale difference magnification processing. Furthermore, by considering the grayscale difference of each pixel in the grayscale difference map, the comprehensiveness and accuracy of the initial damage detection results are further improved.

[0061] S205. If the initial damaged area is located outside the key area of ​​the document, calculate the texture similarity between the initial damaged area and the key area of ​​the document, and filter the initial damaged detection result according to the texture similarity to obtain the target damaged detection result.

[0062] Non-critical areas of a negotiable instrument, i.e., areas outside the critical areas, mainly include edge blank areas and background texture areas. Minor damage in these non-critical areas can easily be confused with the normal texture of the instrument and photographic noise. The texture of the critical areas is relatively fixed and uniform. The normal texture of non-critical areas shows a certain correlation with the texture of the critical areas, while the texture of truly damaged areas differs significantly from that of the critical areas. Therefore, the texture similarity between the initial damaged area and the critical areas can be calculated to determine whether the initial damaged area is truly damaged. Texture similarity measures the consistency of texture features between the initial damaged area and the critical areas.

[0063] Specifically, if the initial damaged area is located outside the key area of ​​the document, the texture similarity between the initial damaged area and the key area of ​​the document is calculated. If the texture similarity is higher than the preset similarity threshold, the initial damaged area is defined as non-damaged, and the corresponding initial damaged detection result is discarded; otherwise, the initial damaged area is determined as real damage, the corresponding initial damaged detection result is retained, and the target damaged detection result is obtained.

[0064] The preset similarity threshold is a pre-defined upper limit for the texture similarity between the initially damaged area and the key area of ​​the document when damage exists. This threshold is used to verify whether the initial document area located outside the key area is damaged. A texture similarity higher than the preset threshold indicates a high degree of consistency between the texture of the initially damaged area and the normal texture of the key area; in this case, the texture of the initially damaged area is normal and can be determined as non-damaged. A texture similarity less than or equal to the preset threshold indicates a low degree of consistency between the texture of the initially damaged area and the normal texture of the key area; in this case, the texture of the initially damaged area is abnormal and can be determined as genuine damage.

[0065] The technical solution of this invention introduces differentiated verification logic by determining whether the initial damaged area in the initial damage detection result is located within the key area of ​​the bill where the key information of the bill is located. Combining the characteristics of the "key area of ​​the bill", the initial damage detection result is differentially verified. Considering that slight damage in the key area of ​​the bill has a significant impact on the validity of the bill, and that the grayscale changes are weak and easy to miss, a grayscale difference amplification filtering method is used for the key area of ​​the bill for accurate verification, highlighting the characteristics of slight damage and improving the accuracy of slight damage bill detection. For non-key areas of the bill, texture similarity filtering is used to eliminate normal texture interference, which can reduce missed detections and false detections, thus doubly ensuring the accuracy of bill damage detection.

[0066] Example 3 Figure 3 This is a schematic diagram of a document damage detection device provided in Embodiment 3 of the present invention. This embodiment of the invention is applicable to detecting documents with different types of damage. The device can execute document damage detection methods and can be implemented in hardware and / or software. The device can be configured in an electronic device that carries document damage detection functions, such as a client or server.

[0067] See Figure 3 The document damage detection device shown includes: a document key information detection module 301, an initial damage detection module 302, and a target damage detection module 303. The document key information detection module 301 is used to determine the target document type and the region of interest (ROI) image of the original document image using a first target detection model, and extract document key information from the ROI image based on the target document type. The initial damage detection module 302 is used to perform damage detection on the ROI image based on the document key information using a second target detection model to obtain an initial damage detection result. The target damage detection module 303 is used to process the initial damage detection result based on the document key information to obtain a target damage detection result; the target damage detection result includes a target damage type and a target damage area.

[0068] The technical solution of this invention employs a first target detection model to determine the target ticket type and the region of interest (ROI) image of the original ticket image. Based on the target ticket type, key ticket information is extracted from the ROI image. Firstly, the first target detection model clarifies the range of the ROI image, eliminating information interference from ticket damage detection, reducing computational power consumption, and improving the real-time performance and accuracy of ticket damage detection. Simultaneously, it extracts key ticket information as a reference, providing data support for subsequent damage detection. Furthermore, based on the key ticket information, a second target detection model is used to perform damage detection on the ROI image, obtaining an initial damage detection result. This initial damage detection result is then processed based on the key ticket information to obtain the target damage detection result. By fusing the key ticket information through a dual-target detection model, the feature extraction of minor damage is enhanced, solving the problem of weak and easily overlooked features of minor damage. This significantly improves the recognition rate of minor ticket damage such as minor creases, light-colored stains, and edge damage, effectively reducing missed detections and improving the accuracy of minor ticket damage detection.

[0069] In an optional embodiment of the present invention, the target damage detection module 303 includes: a key area detection unit for determining whether the initial damaged area in the initial damage detection result is located within the key area of ​​the key information of the document; a grayscale difference amplification and filtering unit for performing grayscale difference amplification processing on the initial damaged area based on the standard lossless texture template of the key area of ​​the document if the initial damaged area is located within the key area of ​​the document, and filtering the initial damage detection result according to the amplified grayscale difference result; and a texture similarity filtering unit for calculating the texture similarity between the initial damaged area and the key area of ​​the document if the initial damaged area is located outside the key area of ​​the document, and filtering the initial damage detection result according to the texture similarity.

[0070] In an optional embodiment of the present invention, the grayscale difference amplification and filtering unit includes: a grayscale difference result detection subunit, used to perform grayscale difference amplification processing on the initial damaged area based on the standard lossless texture template of the key area of ​​the bill, to obtain the amplified grayscale difference result; and an initial damage detection result filtering subunit, used to determine that the initial damaged area is a real damage and retain the corresponding initial damage detection result if the amplified grayscale difference result reaches the damage determination threshold of the corresponding key area of ​​the bill; otherwise, determine that the initial damaged area is not damaged and discard the corresponding initial damage detection result.

[0071] In an optional embodiment of the present invention, the grayscale difference result detection subunit includes: a grayscale difference map generation subunit, used to calculate the grayscale difference between the standard lossless texture template of the key area of ​​the ticket and the pixels in the initial damaged area, and generate a grayscale difference map; and a grayscale difference result detection subunit, used to magnify the grayscale difference map to obtain the magnified grayscale difference result.

[0072] In an optional embodiment of the present invention, the initial damage detection module 302 includes: a feature extraction unit, used to input the image of the region of interest of the bill into the main branch of the second target detection model, and input the key information of the bill into the context branch of the second target detection model to extract features respectively, so as to obtain the main features of the bill and the key context features; a feature fusion unit, used to fuse the main features of the bill and the key context features to obtain fused features; and an initial damage detection unit, used to predict the damage candidate boxes according to the fused features, and to filter the damage candidate boxes to obtain the initial damage detection result.

[0073] In an optional embodiment of the present invention, the initial damage detection unit includes: an initial damage type detection subunit, configured to determine the initial damage type based on the area of ​​the damage candidate box; the initial damage type being either small target damage or non-small target damage; a first damage candidate box filtering subunit, configured to filter the damage candidate box using a first intersection-union (IU) threshold if the initial damage type is small target damage; and a second damage candidate box filtering subunit, configured to filter the damage candidate box using a second IU threshold if the initial damage type is non-small target damage; wherein the first IU threshold is less than the second IU threshold.

[0074] The document damage detection device provided in this embodiment of the invention can execute the document damage detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0075] In the technical solutions of this invention, the information collected is information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0076] Example 4 According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0077] Figure 4A schematic diagram of an electronic device 400 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0078] like Figure 4 As shown, the electronic device 400 includes at least one processor 401 and a memory, such as a read-only memory 402 or a random access memory 403, communicatively connected to the at least one processor 401. The memory stores computer programs executable by the at least one processor. The processor 401 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 402 or loaded from storage unit 408 into the random access memory 403. The random access memory 403 may also store various programs and data required for the operation of the electronic device 400. The processor 401, read-only memory 402, and random access memory 403 are interconnected via a bus 404. An input / output interface 405 is also connected to the bus 404.

[0079] Multiple components in electronic device 400 are connected to input / output interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of monitors, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0080] Processor 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 401 performs the various methods and processes described above, such as the ticket damage detection method.

[0081] In some embodiments, the document damage detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 400 via read-only memory 402 and / or communication unit 409. When the computer program is loaded into random access memory 403 and executed by processor 401, one or more steps of the document damage detection method described above may be performed. Alternatively, in other embodiments, processor 401 may be configured to perform the document damage detection method by any other suitable means (e.g., by means of firmware).

[0082] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0083] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0084] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube or liquid crystal display monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0086] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0087] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system. This addresses the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0088] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0089] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting damaged invoices, characterized in that, The method includes: The first object detection model is used to determine the target bill type and the region of interest (ROI) image of the original bill image, and key bill information is extracted from the ROI image based on the target bill type. Based on the key information of the invoice, a second target detection model is used to perform damage detection on the region of interest image of the invoice to obtain an initial damage detection result. The initial damage detection result is processed based on the key information of the ticket to obtain the target damage detection result; the target damage detection result includes the target damage type and the target damage area.

2. The method according to claim 1, characterized in that, The processing of the initial damage detection result based on the key information of the invoice includes: Determine whether the initial damaged area in the initial damage detection result is located within the key area of ​​the bill where the key information of the bill is located; If the initial damaged area is located within the key area of ​​the bill, then the initial damaged area is subjected to grayscale difference amplification processing based on the standard lossless texture template of the key area of ​​the bill, and the initial damaged detection result is filtered according to the amplified grayscale difference result; If the initial damaged area is located outside the key area of ​​the document, the texture similarity between the initial damaged area and the key area of ​​the document is calculated, and the initial damaged detection result is filtered according to the texture similarity.

3. The method according to claim 2, characterized in that, The method involves amplifying the grayscale difference of the initial damaged area using a standard lossless texture template based on the key area of ​​the document, and filtering the initial damaged detection result based on the amplified grayscale difference result, including: Based on the standard lossless texture template of the key area of ​​the bill, the initial damaged area is subjected to grayscale difference magnification processing to obtain the magnified grayscale difference result. If the magnified grayscale difference result reaches the damage determination threshold of the corresponding key area of ​​the ticket, then the initial damaged area is determined to be a real damage, and the corresponding initial damage detection result is retained; otherwise, the initial damaged area is determined to be non-damaged, and the corresponding initial damage detection result is discarded.

4. The method according to claim 3, characterized in that, The standard lossless texture template based on the key area of ​​the document is used to perform grayscale difference amplification processing on the initial damaged area to obtain the amplified grayscale difference result, including: Calculate the grayscale difference between the standard lossless texture template of the key area of ​​the ticket and the pixels in the initial damaged area to generate a grayscale difference map; The grayscale difference image is magnified to obtain the magnified grayscale difference result.

5. The method according to claim 1, characterized in that, Based on the key information of the invoice, a second object detection model is used to perform damage detection on the region of interest image of the invoice to obtain an initial damage detection result, including: The region of interest image of the bill is input into the main branch of the second target detection model, and the key information of the bill is input into the context branch of the second target detection model for feature extraction, so as to obtain the main features and key context features of the bill. The main features of the invoice and the key context features are fused to obtain fused features; Based on the fusion features, candidate boxes for damage are predicted, and the candidate boxes for damage are filtered to obtain the initial damage detection results.

6. The method according to claim 5, characterized in that, The filtering of the damaged candidate boxes includes: The initial damage type is determined based on the area of ​​the candidate damaged box; the initial damage type is either small target damage or non-small target damage. If the initial damage type is small target damage, then the first intersection-union threshold is used to filter the damage candidate boxes; If the initial damage type is non-small target damage, then the second cross-union ratio threshold is used to filter the damage candidate boxes; wherein, the first cross-union ratio threshold is less than the second cross-union ratio threshold.

7. A device for detecting damaged tickets, characterized in that, The device includes: The bill key information detection module is used to determine the target bill type and the region of interest (ROI) image of the original bill image using a first target detection model, and to extract key bill information from the ROI image based on the target bill type. The initial damage detection module is used to perform damage detection on the region of interest image of the ticket based on the key information of the ticket and using the second target detection model to obtain the initial damage detection result; The target damage detection module is used to process the initial damage detection result based on the key information of the ticket to obtain the target damage detection result; the target damage detection result includes the target damage type and the target damage area.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the document damage detection method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the document damage detection method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the document damage detection method according to any one of claims 1-6.

Images