A training method and device of a mixed paste bill detection model

Abstract

The application discloses a training method and device of a mixed-pasted bill detection model, acquires a mixed-pasted bill sample image, and the mixed-pasted bill sample image comprises a plurality of first bill images with annotated vertex coordinates. Connecting the vertex coordinates of the plurality of first bill images can obtain the bounding boxes of the plurality of first bill images, and the bounding boxes of the plurality of first bill images are shrunk according to a preset ratio to obtain the bounding boxes of a plurality of second bill images. A plurality of target boundary regions between the bounding boxes of the plurality of first bill images and the corresponding bounding boxes of the plurality of second bill images are determined, a detection network is trained according to the mixed-pasted bill image and the plurality of target boundary regions, and a mixed-pasted bill detection model is obtained. It can be seen that the plurality of target boundary regions are used as training labels to train the detection network to obtain the mixed-pasted bill detection model, the extraction capability of the detection network for boundary information of bill images in the mixed-pasted bill sample image is enhanced, and therefore the bill detection accuracy of the mixed-pasted bill detection model is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a training method and apparatus for a mixed-issue ticket detection model. Background Technology

[0002] With the gradual development and popularization of artificial intelligence technology in production and daily life, compared with traditional manual data entry, the application of artificial intelligence technology can effectively improve work efficiency and reduce the training cost of business personnel. As a key part of this, invoice detection technology has also gradually developed.

[0003] In existing technologies, a mixed-issue ticket detection model can be obtained by training a detection model on a ticket image with marked borders in a mixed-issue ticket sample image. The mixed-issue ticket detection model is then used to perform image detection on the mixed-issue ticket image to be detected, thereby obtaining the ticket image in the mixed-issue ticket image to be detected.

[0004] However, in real-world business scenarios, invoice images are often pasted closely together and overlap. The existing mixed-invoice detection models often cannot distinguish between closely pasted and overlapping invoice images, thus failing to accurately detect invoice images and resulting in low invoice detection accuracy. Summary of the Invention

[0005] In view of this, embodiments of this application provide a training method and apparatus for a mixed-issue invoice detection model, aiming to improve the invoice detection accuracy of the mixed-issue invoice detection model.

[0006] In a first aspect, embodiments of this application provide a training method for a mixed-issue invoice detection model, the method comprising:

[0007] Obtain a sample image of mixed-application invoices, wherein the sample image of mixed-application invoices includes multiple first invoice images with labeled vertex coordinates;

[0008] Connect the vertex coordinates of the plurality of first ticket images to obtain the borders of the plurality of first ticket images;

[0009] The borders of the multiple first ticket images are shrunk according to a preset ratio to obtain the borders of the multiple second ticket images;

[0010] Determine multiple target boundary regions between the borders of the plurality of first ticket images and the corresponding borders of the plurality of second ticket images;

[0011] The detection network is trained based on the mixed-issue ticket image and the multiple target boundary regions to obtain a mixed-issue ticket detection model.

[0012] Optionally, training the detection network based on the mixed-issue ticket image and the multiple target boundary regions to obtain a mixed-issue ticket detection model includes:

[0013] The detection network is used to perform image detection on the mixed-issue invoice image to obtain multiple first predicted boundary regions in the mixed-issue invoice image;

[0014] The model parameters of the detection network are trained based on the multiple target boundary regions, the multiple first predicted boundary regions, and the first loss function of the detection network to obtain the mixed-issue ticket detection model.

[0015] Optionally, the detection network includes a series of convolutional layers, normalization layers, and activation layers.

[0016] Optionally, the plurality of first ticket images are labeled with corresponding category labels, and the step of training the detection network based on the mixed ticket images and the plurality of target boundary regions to obtain a mixed ticket detection model specifically involves:

[0017] The detection network is trained based on the mixed-issue ticket image, the multiple target boundary regions, and the multiple category labels to obtain the mixed-issue ticket detection model.

[0018] Optionally, training the detection network based on the mixed-issue ticket image, the multiple target boundary regions, and the multiple category labels to obtain the mixed-issue ticket detection model includes:

[0019] The detection network is used to perform image detection on the mixed-issue invoice image to obtain multiple first predicted boundary regions and corresponding multiple first predicted categories in the mixed-issue invoice image.

[0020] The model parameters of the detection network are trained based on the plurality of target boundary regions, the plurality of first predicted boundary regions, the plurality of category labels, the plurality of first predicted categories, and the second loss function of the detection network to obtain the mixed-issue ticket detection model.

[0021] Secondly, embodiments of this application provide a method for detecting mixed-issue invoices, the method comprising:

[0022] Obtain the image of the mixed-issue tickets to be detected;

[0023] Image detection is performed on the image of mixed-issued receipts to be detected by a mixed-issued receipt detection model to obtain multiple second predicted boundary regions in the image of mixed-issued receipts to be detected. The mixed-issued receipt detection model is obtained according to the training method of the mixed-issued receipt detection model described in any one of the first aspects.

[0024] The third ticket image in the plurality of second predicted boundary regions and the corresponding plurality of second predicted boundary regions are fused to obtain a plurality of fourth ticket images in the mixed ticket image to be detected.

[0025] Optionally, the step of performing image detection on the image of mixed-issued receipts to be detected using a mixed-issued receipt detection model to obtain multiple second predicted boundary regions in the image of mixed-issued receipts to be detected specifically involves:

[0026] The mixed-issue ticket detection model is used to perform image detection on the image of the mixed-issue tickets to be detected, and the multiple second prediction boundary regions and the corresponding multiple second prediction categories are obtained. The mixed-issue ticket detection model is obtained according to the training method of the mixed-issue ticket detection model described in any one of the fourth to fifth items of the first aspect.

[0027] The step of fusing the third ticket image in the plurality of second predicted boundary regions with the corresponding plurality of second predicted boundary regions to obtain a plurality of fourth ticket images in the mixed ticket image to be detected specifically involves:

[0028] The third ticket image in the plurality of second prediction boundary regions, the corresponding plurality of second prediction boundary regions, and the corresponding plurality of second prediction categories are fused to obtain the plurality of fourth ticket images.

[0029] Thirdly, embodiments of this application provide a training apparatus for a mixed-issue invoice detection model, the apparatus comprising:

[0030] The first acquisition module is used to acquire mixed-application invoice sample images, which include multiple first invoice images with labeled vertex coordinates;

[0031] A connecting module is used to connect the vertex coordinates of the plurality of first ticket images to obtain the borders of the plurality of first ticket images;

[0032] The shrinking module is used to shrink the borders of the plurality of first ticket images according to a preset ratio to obtain the borders of the plurality of second ticket images.

[0033] The determining module is used to determine multiple target boundary regions between the borders of the plurality of first ticket images and the corresponding borders of the plurality of second ticket images;

[0034] The training module is used to train the detection network based on the mixed-issue ticket image and the multiple target boundary regions to obtain a mixed-issue ticket detection model.

[0035] Fourthly, embodiments of this application provide a detection device for mixed-issue invoices, characterized in that the device comprises:

[0036] The second acquisition module is used to acquire images of mixed-issue tickets to be detected;

[0037] The image detection module is used to perform image detection on the image of mixed-issued receipts to be detected using a mixed-issued receipt detection model, and to obtain multiple second predicted boundary regions in the image of mixed-issued receipts to be detected. The mixed-issued receipt detection model is obtained according to the training method of the mixed-issued receipt detection model in any of the first aspects.

[0038] The fusion module is used to fuse the third ticket image in the plurality of second predicted boundary regions with the corresponding plurality of second predicted boundary regions to obtain a plurality of fourth ticket images in the mixed ticket image to be detected.

[0039] Fifthly, embodiments of this application provide an apparatus, the apparatus comprising:

[0040] Memory, used to store computer programs;

[0041] A processor is configured to execute the computer program to cause the device to perform the training method of the mixed-issue ticket detection model according to any one of the first aspects, or the mixed-issue ticket detection method according to any one of the second aspects.

[0042] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the training method for the mixed-issue ticket detection model described in any of the first aspects, or the mixed-issue ticket detection method described in any of the second aspects.

[0043] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0044] This application provides a training method and apparatus for a mixed-issue invoice detection model. First, a mixed-issue invoice sample image is acquired, comprising multiple first invoice images labeled with vertex coordinates. Then, the vertex coordinates of the multiple first invoice images are concatenated to obtain the bounding boxes of the multiple first invoice images. These bounding boxes are then shrunk according to a preset ratio to obtain the bounding boxes of multiple second invoice images. Finally, multiple target boundary regions are determined between the bounding boxes of the multiple first invoice images and the corresponding bounding boxes of the multiple second invoice images. A detection network is trained based on the mixed-issue invoice images and these target boundary regions to obtain the mixed-issue invoice detection model. It is evident that using the bounding boxes of the multiple first invoice images in the mixed-issue invoice sample image, and the boundary regions between them and the bounding boxes of the multiple second invoice images obtained by shrinking the first invoice images, as training labels for the mixed-issue invoice sample image to train the detection network and obtain the mixed-issue invoice detection model, can enhance the detection network's ability to extract boundary information of the invoice images in the mixed-issue invoice sample image, thereby improving the invoice detection accuracy of the mixed-issue invoice detection model. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the system framework involved in one application scenario in the embodiments of this application;

[0047] Figure 2 A flowchart illustrating a method for training a mixed-issue ticket detection model provided in this application embodiment;

[0048] Figure 3 A flowchart illustrating a method for detecting mixed-issue invoices provided in this application embodiment;

[0049] Figure 4 This is a schematic diagram of the structure for image detection using a mixed-issue ticket detection model provided in an embodiment of this application;

[0050] Figure 5 A schematic diagram of the structure of a training device for a mixed-issue ticket detection model provided in an embodiment of this application;

[0051] Figure 6 This is a schematic diagram of a device for detecting mixed-issue tickets provided in an embodiment of this application. Detailed Implementation

[0052] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0053] In existing technologies, a mixed-issue ticket detection model can be obtained by training a detection model on a ticket image with marked borders in a mixed-issue ticket sample image. The mixed-issue ticket detection model is then used to perform image detection on the mixed-issue ticket image to be detected, thereby obtaining the ticket image in the mixed-issue ticket image to be detected.

[0054] However, in real-world business scenarios, invoice images are often pasted closely together and overlap. The existing mixed-invoice detection models often cannot distinguish between closely pasted and overlapping invoice images, thus failing to accurately detect invoice images and resulting in low invoice detection accuracy.

[0055] To address the aforementioned problems, this application provides a training method and apparatus for a mixed-issue ticket detection model. First, mixed-issue ticket sample images are acquired, including multiple first ticket images with labeled vertex coordinates. Then, the vertex coordinates of the multiple first ticket images are connected to obtain the bounding boxes of the multiple first ticket images. These bounding boxes are then shrunk according to a preset ratio to obtain the bounding boxes of multiple second ticket images. Finally, multiple target boundary regions are determined between the bounding boxes of the multiple first ticket images and the corresponding bounding boxes of the multiple second ticket images. The detection network is trained based on the mixed-issue ticket images and the multiple target boundary regions to obtain a mixed-issue ticket detection model.

[0056] It is evident that using the boundary regions between the borders of multiple first ticket images in the mixed ticket sample image and the borders of multiple second ticket images obtained by shrinking the multiple first ticket images as training labels to train the detection network to obtain the mixed ticket detection model can enhance the detection network's ability to extract boundary information of ticket images in the mixed ticket sample image, thereby improving the ticket detection accuracy of the mixed ticket detection model.

[0057] For example, one scenario in the embodiments of this application can be applied to, such as Figure 1 The scenario shown includes a camera 101 and a server 102. The camera 101 captures images of mixed-issue tickets and sends them to the server 102. The server 102 uses the implementation method provided in this application to train the mixed-issue ticket detection model and detect mixed-issue tickets.

[0058] First, in the above application scenarios, although the action descriptions of the implementation methods provided in this application are executed by the server 102, the implementation methods of this application are not limited in terms of the execution subject, as long as the actions disclosed in the implementation methods provided in this application are executed.

[0059] Secondly, the above scenario is only one example provided by the embodiments of this application, and the embodiments of this application are not limited to this scenario.

[0060] The following, in conjunction with the accompanying drawings, describes in detail the specific implementation of the training method and apparatus for the mixed-issue ticket detection model in the embodiments of this application through examples.

[0061] Figure 2 This document presents a flowchart illustrating a training method for a mixed-issue ticket detection model, as provided in an embodiment of this application. Figure 2 As shown in the embodiments of this application, the training method for the mixed-issue ticket detection model may include:

[0062] S201: Obtain a mixed-issue sample image, which includes multiple first-issue images with labeled vertex coordinates.

[0063] Mixed document images refer to multiple document images pasted onto a single background image. The process involves obtaining a sample mixed document image, which contains multiple first document images with labeled vertex coordinates.

[0064] S202: Connect the vertex coordinates of multiple first ticket images to obtain the borders of multiple first ticket images.

[0065] S203: Shrink the borders of multiple first ticket images according to a preset ratio to obtain the borders of multiple second ticket images.

[0066] The borders of multiple first ticket images are shrunk inward by a preset ratio to obtain the borders of multiple second ticket images. As one possible implementation, the indentation coefficient of the preset indentation ratio can be 0.95, which gives the mixed ticket model a clear learning target.

[0067] S204: Determine multiple target boundary regions between the borders of multiple first ticket images and the borders of multiple corresponding second ticket images.

[0068] Multiple target boundary regions refer to the regions between the borders of multiple first ticket images and the borders of the corresponding multiple second ticket images. Multiple target boundary regions can be determined based on the borders of the multiple first ticket images and the corresponding borders of the multiple second ticket images.

[0069] S205: Train the detection network based on the mixed-issue ticket images and multiple target boundary regions to obtain a mixed-issue ticket detection model.

[0070] Image detection is performed on mixed-issue document images using a detection network. Multiple first predicted boundary regions are obtained in the mixed-issue document images. Then, based on the multiple target boundary regions, the multiple first predicted boundary regions, and the first loss function of the detection network, the model parameters of the detection network are trained to obtain a mixed-issue document detection model. The detection network includes concatenated convolutional layers, normalization layers, and activation layers. Of course, this application does not specifically limit the structure of the detection network and does not affect the implementation of the embodiments of this application.

[0071] The above describes a training method for a mixed-issue invoice detection model provided in this application embodiment. First, a mixed-issue invoice sample image is acquired, comprising multiple first invoice images labeled with vertex coordinates. Then, the vertex coordinates of the multiple first invoice images are connected to obtain the bounding boxes of the multiple first invoice images. These bounding boxes are then shrunk according to a preset ratio to obtain the bounding boxes of multiple second invoice images. Finally, multiple target boundary regions are determined between the bounding boxes of the multiple first invoice images and the corresponding bounding boxes of the multiple second invoice images. The detection network is trained based on the mixed-issue invoice images and the multiple target boundary regions to obtain a mixed-issue invoice detection model. It is evident that using the bounding boxes of the multiple first invoice images in the mixed-issue invoice sample image, and the boundary regions between them and the bounding boxes of the multiple second invoice images obtained by shrinking the multiple first invoice images, as training labels for the mixed-issue invoice sample image to train the detection network and obtain the mixed-issue invoice detection model, can enhance the detection network's ability to extract boundary information of the invoice images in the mixed-issue invoice sample image, thereby improving the invoice detection accuracy of the mixed-issue invoice detection model.

[0072] In this embodiment of the application, S205 may specifically include the following S2051-S2052:

[0073] S2051: Perform image detection on the mixed-issue ticket image using a detection network to obtain multiple first-prediction boundary regions and corresponding multiple first-prediction categories in the mixed-issue ticket image.

[0074] By using a detection network to perform image detection on mixed-issue invoice images, multiple first predicted boundary regions and corresponding predicted categories of multiple first invoice images are obtained in the mixed-issue invoice images.

[0075] S2052: Based on multiple target boundary regions, multiple first predicted boundary regions, multiple category labels, multiple first predicted categories, and the second loss function of the detection network, train the model parameters of the detection network to obtain the mixed-issue ticket detection model.

[0076] Multiple first ticket images are labeled with corresponding category labels, such as invoices or tickets. Based on multiple target boundary regions, multiple first predicted boundary regions, multiple label categories, the predicted categories of multiple first ticket images, and the second loss function of the detection network, the model parameters of the detection network are trained to obtain a mixed ticket detection model. In this model, the multiple target boundary regions and multiple label categories are pre-labeled.

[0077] By performing image detection on mixed-item ticket images using a detection network, multiple first predicted boundary regions and corresponding first predicted categories can be obtained. Then, based on these multiple target boundary regions, first predicted boundary regions, category labels, first predicted categories, and the detection network's second loss function, the model parameters of the detection network are trained to obtain a mixed-item ticket detection model. It is evident that using the boundary regions between the bounding boxes of multiple first ticket images and the bounding boxes of multiple second ticket images obtained by shrinking the first ticket images, along with the categories of the first ticket images, as training labels for the mixed-item ticket sample image, to train the detection network and obtain the mixed-item ticket detection model, enhances the detection network's ability to extract boundary and category information from the ticket images in the mixed-item ticket sample image, thereby improving the ticket detection accuracy of the mixed-item ticket detection model.

[0078] Figure 3 This is a flowchart illustrating a method for detecting mixed-issue invoices, provided in an embodiment of this application. (Combined with...) Figure 3 As shown in the embodiments of this application, the method for detecting mixed-issue invoices may include:

[0079] S301: Obtain the image of the mixed-issue tickets to be detected.

[0080] S302: The mixed-issue ticket detection model is used to perform image detection on the mixed-issue ticket image to be detected, and multiple second predicted boundary regions in the mixed-issue ticket image are obtained. The mixed-issue ticket detection model is obtained according to the training method of the mixed-issue ticket detection model provided in the above-mentioned application embodiment.

[0081] According to the training method of the mixed-issue ticket detection model provided in the above application embodiment, a trained mixed-issue ticket detection model can be obtained. Then, image detection is performed on the mixed-issue ticket image to be detected based on the mixed-issue ticket detection model to obtain multiple second predicted boundary regions of the mixed-issue ticket image to be detected.

[0082] S303: Fuse the third ticket image in multiple second predicted boundary regions with the corresponding multiple second predicted boundary regions to obtain multiple fourth ticket images in the mixed ticket image to be detected.

[0083] The third ticket image in multiple second prediction boundary regions refers to the image obtained after shrinking multiple first ticket images in the mixed ticket image to be detected according to a preset ratio. Then, the third ticket image in multiple second prediction boundary regions is fused with the corresponding multiple second prediction boundary regions to obtain multiple fourth ticket images in the mixed ticket image to be detected.

[0084] The above describes a method for detecting mixed-issue tickets according to an embodiment of this application. First, an image of the mixed-issue tickets to be detected is acquired. Then, an image detection model is used to detect the mixed-issue tickets in the image, resulting in multiple second predicted boundary regions. Finally, the third ticket image within the multiple second predicted boundary regions is fused with the corresponding multiple second predicted boundary regions to obtain multiple fourth ticket images within the mixed-issue ticket image. It is evident that fusing the multiple second predicted boundary regions obtained by the mixed-issue ticket detection model with the third ticket images within those regions yields multiple fourth ticket images of the mixed-issue ticket image with clearer boundary information, enabling the training effect of the mixed-issue ticket detection model to achieve pixel-level confidence.

[0085] In this embodiment of the application, S302 may specifically include the following S3021-S3022:

[0086] S3021: The mixed-issue ticket detection model is used to perform image detection on the mixed-issue ticket image to be detected, and multiple second prediction boundary regions and corresponding multiple second prediction categories are obtained. The mixed-issue ticket detection model is obtained according to the training method of the mixed-issue ticket detection model provided in the above-mentioned application embodiment.

[0087] Through the steps S2051-S2052 in the above application embodiment, a trained mixed-issue ticket detection model is obtained. Then, image detection is performed on the mixed-issue ticket image to be detected according to the mixed-issue ticket detection model to obtain multiple second prediction boundary regions and corresponding multiple second prediction categories.

[0088] S3022: Fuse the third ticket image in multiple second prediction boundary regions, the corresponding multiple second prediction boundary regions, and the corresponding multiple second prediction categories to obtain multiple fourth ticket images.

[0089] As one possible implementation, a schematic diagram of the structure for image detection of mixed-issue invoices using a mixed-issue invoice detection model can be found in [reference needed]. Figure 4The diagram illustrates, for example, that the image of the mixed-issue ticket to be detected is processed by the backbone network BAKCBone for feature extraction, then by a feature pyramid network (FPN) for multi-level feature extraction, and then by an upsampling CBL module that outputs three branches. Each CBL module includes a cascaded normalization layer, a convolutional layer, and an activation layer. Boundary region information is extracted using the BoarderNet boundary extraction structure. The boundary region information extracted by the BoarderNet is fused with the ticket image information extracted by the regression detection target Bbox. Then, the Bbox branch after fusing the ticket image information and the category information obtained by the classification CLS branch are fed into the feature alignment structure MergeNet for category information fusion. Specifically, the third ticket image in multiple second predicted boundary regions, the corresponding multiple second predicted boundary regions, and the corresponding multiple second predicted categories are fused to obtain multiple fourth ticket images.

[0090] A mixed-issue ticket detection model is used to detect mixed-issue tickets in the image, obtaining multiple second-predicted boundary regions and corresponding second-predicted categories. Then, the third ticket image within each of the multiple second-predicted boundary regions, along with the corresponding second-predicted boundary regions and their respective second-predicted categories, are fused to obtain multiple fourth ticket images. Fusing the detected second-predicted categories and their corresponding second-predicted boundary regions with the segmented third ticket images within the second-predicted boundary regions aligns the features, thereby improving the regression accuracy of mixed-issue ticket detection.

[0091] The above describes some specific implementations of the training method for the mixed-issue document detection model and the detection method for mixed-issue documents provided in this application. Based on this, this application also provides a corresponding apparatus. The apparatus provided in this application will be described below from the perspective of functional modularity.

[0092] See Figure 5 The figure is a schematic diagram of the structure of a training device 500 for a mixed-issue ticket detection model provided in an embodiment of this application. The device 500 may include:

[0093] The first acquisition module 501 is used to acquire a mixed-issue sample image, which includes multiple first issue images with labeled vertex coordinates;

[0094] The connecting module 502 is used to connect the vertex coordinates of multiple first ticket images to obtain the borders of multiple first ticket images;

[0095] The shrinking module 503 is used to shrink the borders of multiple first ticket images according to a preset ratio to obtain the borders of multiple second ticket images.

[0096] The determination module 504 is used to determine multiple target boundary regions between the borders of multiple first ticket images and the borders of multiple corresponding second ticket images;

[0097] Training module 505 is used to train the detection network based on mixed-issue ticket images and multiple target boundary regions to obtain a mixed-issue ticket detection model.

[0098] In this embodiment, the five modules—first acquisition module 501, connection module 502, shrinking module 503, determination module 504, and training module 505—cooperate to train the detection network using the boundary regions between the borders of multiple first ticket images in the mixed ticket sample image and the borders of multiple second ticket images obtained by shrinking the multiple first ticket images as training labels for the mixed ticket sample image. This enhances the detection network's ability to extract boundary information of ticket images in the mixed ticket sample image, thereby improving the ticket detection accuracy of the mixed ticket detection model.

[0099] As one implementation method, the training module 505 may specifically include:

[0100] An image detection unit is used to perform image detection on a mixed-issue ticket image through a detection network to obtain multiple first predicted boundary regions in the mixed-issue ticket image.

[0101] The training unit is used to train the model parameters of the detection network based on multiple target boundary regions, multiple first predicted boundary regions, and the first loss function of the detection network, to obtain the mixed-issue ticket detection model.

[0102] As one implementation, the detection network includes convolutional layers, normalization layers, and activation layers connected in series.

[0103] As one implementation method, the training module 505 is specifically used for:

[0104] A mixed-issue ticket detection model is obtained by training a detection network based on mixed-issue ticket images, multiple target boundary regions, and multiple category labels.

[0105] As one implementation method, the training module 505 is specifically used for:

[0106] Image detection is performed on the mixed-issue invoice image by a detection network to obtain multiple first-predicted boundary regions and corresponding multiple first-predicted categories in the mixed-issue invoice image;

[0107] Based on multiple target boundary regions, multiple first predicted boundary regions, multiple category labels, multiple first predicted categories, and the second loss function of the detection network, the model parameters of the detection network are trained to obtain the mixed-issue ticket detection model.

[0108] See Figure 6 The figure is a schematic diagram of the structure of a mixed-issue ticket detection device 600 provided in an embodiment of this application. The device 600 may include:

[0109] The second acquisition module 601 is used to acquire the image of the mixed-issue tickets to be detected;

[0110] Image detection module 602 is used to perform image detection on the image of mixed-issued invoices to be detected using a mixed-issued invoice detection model, and to obtain multiple second predicted boundary regions in the image of mixed-issued invoices to be detected. The mixed-issued invoice detection model is obtained by the training device 500 of the mixed-issued invoice detection model according to the above-mentioned application embodiment.

[0111] The fusion module 603 is used to fuse the third ticket image in multiple second prediction boundary regions with the corresponding multiple second prediction boundary regions to obtain multiple fourth ticket images in the mixed ticket image to be detected.

[0112] In this embodiment of the application, through the cooperation of the second acquisition module 601, the image detection module 602 and the fusion module 603, the multiple second predicted boundary regions and the third ticket images in the multiple second predicted boundary regions obtained by the mixed ticket detection model are fused to obtain multiple fourth ticket images of the mixed ticket image to be detected, which have clearer boundary information, so that the training effect of the mixed ticket detection model has pixel-level confidence.

[0113] As one implementation method, the image detection module 602 is specifically used for:

[0114] The mixed-issue invoice detection model is used to perform image detection on the mixed-issue invoice image to be detected, and obtain multiple second prediction boundary regions and corresponding multiple second prediction categories. The mixed-issue invoice detection model is obtained by the training device 500 of the mixed-issue invoice detection model according to the above-mentioned application embodiment.

[0115] The third ticket image in multiple second prediction boundary regions, the corresponding multiple second prediction boundary regions, and the corresponding multiple second prediction categories are fused to obtain multiple fourth ticket images.

[0116] This application also provides corresponding devices and computer-readable storage media for implementing the solutions provided in this application.

[0117] The device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to enable the device to perform the training method for the mixed-issue ticket detection model described in any embodiment of this application, or the mixed-issue ticket detection method described in any embodiment of this application.

[0118] The computer-readable storage medium stores a computer program. When the computer program is run, the device running the computer program implements the training method of the mixed-issue ticket detection model described in any embodiment of this application, or the mixed-issue ticket detection method described in any embodiment of this application.

[0119] In the embodiments of this application, the terms "first" and "second" (if they exist) are used only as name identifiers and do not represent the order of first and second.

[0120] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0121] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0122] The above description is merely an exemplary implementation of this application and is not intended to limit the scope of protection of this application.

Claims

1. A method for training a mixed paste bill detection model, characterized in that, The method comprises: obtaining a mixed bill sample image, the mixed bill sample image comprising a plurality of first bill images with annotated vertex coordinates; connecting the vertex coordinates of the plurality of first bill images to obtain a plurality of first bill image bounding boxes; scaling the plurality of first bill image bounding boxes according to a preset ratio to obtain a plurality of second bill image bounding boxes; determining a plurality of target boundary regions between the plurality of first bill image bounding boxes and the corresponding plurality of second bill image bounding boxes; training a detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model; the training of the detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model comprises: performing image detection on the mixed bill sample image by the detection network to obtain a plurality of first predicted boundary regions in the mixed bill sample image; training model parameters of the detection network according to the plurality of target boundary regions, the plurality of first predicted boundary regions and a first loss function of the detection network to obtain the mixed bill detection model; the plurality of first bill images are annotated with a plurality of corresponding class labels, and the training of the detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model specifically comprises: training the detection network according to the mixed bill sample image, the plurality of target boundary regions and the plurality of class labels to obtain the mixed bill detection model; the training of the detection network according to the mixed bill sample image, the plurality of target boundary regions and the plurality of class labels to obtain the mixed bill detection model comprises: performing image detection on the mixed bill sample image by the detection network to obtain a plurality of first predicted boundary regions and a plurality of corresponding first predicted classes in the mixed bill sample image; training model parameters of the detection network according to the plurality of target boundary regions, the plurality of first predicted boundary regions, the plurality of class labels, the plurality of first predicted classes and a second loss function of the detection network to obtain the mixed bill detection model.

2. The method of claim 1, wherein, The detection network comprises a convolution layer, a normalization layer and an activation layer connected in series.

3. A method of detecting a mixed instrument, characterized by, The method comprises: obtaining a mixed bill sample image, the mixed bill sample image comprising a plurality of first bill images with annotated vertex coordinates; connecting the vertex coordinates of the plurality of first bill images to obtain a plurality of first bill image bounding boxes; scaling the plurality of first bill image bounding boxes according to a preset ratio to obtain a plurality of second bill image bounding boxes; determining a plurality of target boundary regions between the plurality of first bill image bounding boxes and the corresponding plurality of second bill image bounding boxes; training a detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model; the training of the detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model comprises: performing image detection on the mixed bill sample image by the detection network to obtain a plurality of first predicted boundary regions in the mixed bill sample image; training model parameters of the detection network according to the plurality of target boundary regions, the plurality of first predicted boundary regions and a first loss function of the detection network to obtain the mixed bill detection model; the plurality of first bill images are annotated with a plurality of corresponding class labels, and the training of the detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model specifically comprises: training the detection network according to the mixed bill sample image, the plurality of target boundary regions and the plurality of class labels to obtain the mixed bill detection model; the training of the detection network according to the mixed bill sample image, the plurality of target boundary regions and the plurality of class labels to obtain the mixed bill detection model comprises: performing image detection on the mixed bill sample image by the detection network to obtain a plurality of first predicted boundary regions and a plurality of corresponding first predicted classes in the mixed bill sample image; training model parameters of the detection network according to the plurality of target boundary regions, the plurality of first predicted boundary regions, the plurality of class labels, the plurality of first predicted classes and a second loss function of the detection network to obtain the mixed bill detection model. The detection network comprises a convolution layer, a normalization layer and an activation layer connected in series. The method comprises: obtaining a mixed bill sample image, the mixed bill sample image comprising a plurality of first bill images with annotated vertex coordinates; connecting the vertex coordinates of the plurality of first bill images to obtain a plurality of first bill image bounding boxes; scaling the plurality of first bill image bounding boxes according to a preset ratio to obtain a plurality of second bill image bounding boxes; determining a plurality of target boundary regions between the plurality of first bill image bounding boxes and the corresponding plurality of second bill image bounding boxes; training a detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model; the training of the detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model comprises: performing image detection on the mixed bill sample image by the detection network to obtain a plurality of first predicted boundary regions in the mixed bill sample image; training model parameters of the detection network according to the plurality of target boundary regions, the plurality of first predicted boundary regions and a first loss function of the detection network to obtain the mixed bill detection model; the plurality of first bill images are annotated with a plurality of corresponding class labels, and the training of the detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model specifically comprises: training the detection network according to the mixed bill sample image, the plurality of target boundary regions and the plurality of class labels to obtain the mixed bill detection model; the training of the detection network according to the mixed bill sample image, the plurality of target boundary regions and the plurality of class labels to obtain the mixed bill detection model comprises: performing image detection on the mixed bill sample image by the detection network to obtain a plurality of first predicted boundary regions and a plurality of corresponding first predicted classes in the mixed bill sample image; training model parameters of the detection network according to the plurality of target boundary regions, the plurality of first predicted boundary regions, the plurality of class labels, the plurality of first predicted classes and a second loss function of the detection network to obtain the mixed bill detection model. The detection network comprises a convolution layer, a normalization layer and an activation layer connected in series. The method comprises: obtaining a mixed bill sample image, the mixed bill sample image comprising a plurality of first bill images with annotated vertex coordinates; connecting the vertex coordinates of the plurality of first bill images to obtain a plurality of first bill image bounding boxes; scaling the plurality of first bill image bounding boxes according to a preset ratio to obtain a plurality of second bill image bounding boxes; determining a plurality of target boundary regions between the plurality of first bill image bounding boxes and the corresponding plurality of second bill image bounding boxes; training a detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model; the training of the detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model comprises: performing image detection on the mixed bill sample image by the detection network to obtain a plurality of first predicted boundary regions in the mixed bill sample image; training model parameters of the detection network according to the plurality of target boundary regions, the plurality of first predicted boundary regions and a first loss function of the detection network to obtain the mixed bill detection model; the plurality of first bill images are annotated with a plurality of corresponding class labels, and the training of the detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model specifically comprises: training the detection network according to the mixed bill sample image, the plurality of target boundary regions and the plurality of class labels to obtain the mixed bill detection model; the training of the detection network according to the mixed bill sample image, the plurality of target boundary regions and the plurality of class labels to obtain the mixed bill detection model comprises: performing image detection on the mixed bill sample image by the detection network to obtain a plurality of first predicted boundary regions and a plurality of corresponding first predicted classes in the mixed bill sample image; training model parameters of the detection network according to the plurality of target boundary regions, the plurality of first predicted boundary regions, the plurality of class labels, the plurality of first predicted classes and a second loss function of the detection network to obtain the mixed bill detection model. The detection network comprises a convolution layer, a normalization layer and an activation layer connected in series. obtaining the plurality of second prediction boundary regions and the corresponding plurality of second prediction categories by performing image detection on the to-be-detected mixed bill image through the mixed bill detection model, the mixed bill detection model being obtained according to the training method of the mixed bill detection model in any one of claims 1; fusing the third bill image in the plurality of second prediction boundary regions and the corresponding plurality of second prediction boundary regions to obtain a plurality of fourth bill images in the to-be-detected mixed bill image, specifically by: fusing the third bill image in the plurality of second prediction boundary regions, the corresponding plurality of second prediction boundary regions, and the corresponding plurality of second prediction categories to obtain the plurality of fourth bill images. 4.A device for training a mixed paste bill detection model, characterized in that, The device comprises: a first acquisition module configured to acquire a mixed bill sample image, the mixed bill sample image comprising a plurality of first bill images with annotated vertex coordinates; a connection module configured to connect the vertex coordinates of the plurality of first bill images to obtain a plurality of first bill image bounding boxes; a contraction module configured to contract the plurality of first bill image bounding boxes according to a preset ratio to obtain a plurality of second bill image bounding boxes; a determination module configured to determine a plurality of target boundary regions between the plurality of first bill image bounding boxes and the corresponding plurality of second bill image bounding boxes; a training module configured to train a detection network according to the mixed bill sample image and the plurality of target boundary regions to obtain a mixed bill detection model; the training module specifically comprises: an image detection unit configured to perform image detection on the mixed bill sample image through the detection network to obtain a plurality of first prediction boundary regions in the mixed bill sample image; a training unit configured to train model parameters of the detection network according to the plurality of target boundary regions, the plurality of first prediction boundary regions, and a first loss function of the detection network to obtain the mixed bill detection model; the training module is specifically configured to: train the detection network according to the mixed bill sample image, the plurality of target boundary regions, and a plurality of category labels to obtain the mixed bill detection model; the training module is specifically configured to: perform image detection on the mixed bill sample image through the detection network to obtain a plurality of first prediction boundary regions in the mixed bill sample image and a plurality of corresponding first prediction categories; train model parameters of the detection network according to the plurality of target boundary regions, the plurality of first prediction boundary regions, the plurality of category labels, the plurality of first prediction categories, and a second loss function of the detection network to obtain the mixed bill detection model.

5. A device for detecting a forgery of a document, characterized in that The device comprises: a second acquisition module configured to acquire a to-be-detected mixed bill image; an image detection module configured to perform image detection on the to-be-detected mixed bill image through a mixed bill detection model to obtain a plurality of second prediction boundary regions in the to-be-detected mixed bill image, the mixed bill detection model being obtained according to the training method of the mixed bill detection model in any one of claims 1 to 2; a fusion module, configured to fuse third bill images in the plurality of second predicted boundary regions and corresponding second predicted boundary regions to obtain a plurality of fourth bill images in the bill image to be detected for collage; the image detection module is specifically configured to: perform image detection on the bill image to be detected for collage by using the bill detection model for collage to obtain a plurality of second predicted boundary regions and a plurality of second predicted categories corresponding to the second predicted boundary regions, wherein the bill detection model for collage is obtained according to the training device of the bill detection model for collage; fuse third bill images in the plurality of second predicted boundary regions, corresponding second predicted boundary regions and corresponding second predicted categories to obtain a plurality of fourth bill images.

6. A document inspection apparatus for detecting a pasted document, characterized by comprising: The device comprises: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the training method of the bill detection model for collage according to any one of claims 1 to 2, or the steps of the detection method of the bill for collage according to any one of claim 3.

Images