Trade document key information detection method and device, equipment and storage medium
By segmenting and feature fusion processing of trade document images, the detection capability for small targets is improved by using a target detection model. This solves the problems of misjudgment and inefficiency caused by manual verification in existing technologies, and achieves efficient and accurate detection of key information in trade documents.
Patent Information
- Application Number
- CN202511048682.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-29
- Publication Date
- 2025-11-14
AI Technical Summary
In existing technologies, the detection of key information in trade documents relies on manual verification, which results in a large workload, low efficiency, and a high risk of misjudgment or omission, affecting the accuracy and timeliness of financing decisions.
A target detection model is used to segment trade document images. Multi-scale feature maps are extracted through a backbone network, and feature fusion and coordinate attention processing are performed by combining a neck network and a detection head to improve the detection capability of small targets and the ability to capture regional directional features.
It improves the accuracy and efficiency of key information detection in trade documents, adapts to image data with strong directionality such as trade documents, enhances the detection capability for small targets, and improves the accuracy and robustness of key information detection.
Smart Images
Figure CN120954032A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and more particularly to the field of image processing, specifically to a method, apparatus, electronic device, and storage medium for detecting key information in trade documents. Background Technology
[0002] Supply chain finance revolves around core enterprises within a supply chain, providing financial services to these core enterprises and their upstream and downstream suppliers based on supply chain relationships. Supply chain financing offers financing services based on the actual transaction processes and business relationships within trade activities, with the trade background providing tangible support for the financing. For example, trade documents such as trade contracts, orders, invoices, and shipping documents record basic transaction information. These documents not only reflect the genuine transaction relationship between the parties but also serve as crucial evidence for credit assessment of financing clients and mitigating financing risks. Currently, the review of trade background mainly relies on manual verification and data entry. This method is not only labor-intensive and inefficient but also prone to misjudgments or omissions due to subjective factors, thus affecting the accuracy and timeliness of financing decisions.
[0003] Therefore, how to detect key information in trade documents has become a critical technical problem that urgently needs to be solved. Summary of the Invention
[0004] This application provides a method, apparatus, electronic device, and storage medium for detecting key information in trade documents, so as to improve the accuracy and efficiency of key information detection.
[0005] In a first aspect, embodiments of this application provide a method for detecting key information in trade documents, including:
[0006] The trade document image is segmented into multiple image slices;
[0007] The image slices are input into the backbone network of the target detection model, and multi-scale feature maps are extracted through the backbone network; the multi-scale feature maps include a second layer feature map;
[0008] The multi-scale feature map is input into the neck network of the target detection model. The neck network fuses the multi-scale feature map to generate an original fused feature map. The original fused feature map is then subjected to coordinate attention processing to obtain an enhanced feature map.
[0009] The enhanced feature map is input into the detection head of the target detection model, and the predicted bounding box and the predicted classification result are output.
[0010] Based on the prediction box and the prediction classification result, determine the key information in the trade document image;
[0011] The detection head includes a second detection branch for processing the second layer feature map, and the second detection branch is connected to a feature fusion structure in the neck network that is adapted to the second layer feature map.
[0012] Secondly, embodiments of this application also provide a device for detecting key information in trade documents, including:
[0013] The image slicing module is used to segment trade document images into multiple image slices;
[0014] The feature extraction module is used to input the image slices into the backbone network of the target detection model, and extract multi-scale feature maps through the backbone network; the multi-scale feature maps include a second layer feature map;
[0015] The feature fusion module is used to input the multi-scale feature map into the neck network of the target detection model, fuse the multi-scale feature map through the neck network to generate an original fused feature map, and perform coordinate attention processing on the original fused feature map to obtain an enhanced feature map.
[0016] The prediction output module is used to input the enhanced feature map into the detection head of the target detection model and output the prediction box and the prediction classification result;
[0017] The key information determination module is used to determine key information in the trade document image based on the prediction box and the prediction classification result;
[0018] The detection head includes a second detection branch for processing the second layer feature map, and the second detection branch is connected to a feature fusion structure in the neck network that is adapted to the second layer feature map.
[0019] Thirdly, embodiments of this application also provide an electronic device, which includes:
[0020] One or more processors;
[0021] Storage device for storing one or more programs;
[0022] When one or more programs are executed by one or more processors, the one or more processors implement any of the key information detection methods for trade documents provided in the embodiments of this application.
[0023] Fourthly, embodiments of this application also provide a storage medium including computer-executable instructions, which, when executed by a computer processor, are used to perform any of the trade document key information detection methods provided in embodiments of this application.
[0024] Fifthly, embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements any of the key information detection methods for trade documents provided in embodiments of this application.
[0025] This application enhances the model's ability to detect small targets by adding a second feature extraction layer with higher resolution to the target detection model, thereby improving the accuracy of key information detection. Furthermore, by segmenting the trade document image into multiple image slices and introducing a coordinate attention mechanism, the model's ability to capture regional directional features is improved, making it more suitable for image data with strong directionality, such as trade documents, and further enhancing the accuracy of key information detection.
[0026] Therefore, the technical solution of this application solves the problem and achieves the desired effect. Attached Figure Description
[0027] Figure 1a This is a flowchart of a method for detecting key information in trade documents according to Embodiment 1 of this application;
[0028] Figure 1b This is a schematic diagram of the structure of a target detection model provided according to Embodiment 1 of this application;
[0029] Figure 2a This is a flowchart of another method for detecting key information in trade documents according to Embodiment 2 of this application;
[0030] Figure 2b This is a schematic diagram illustrating the principle of a coordinate attention mechanism according to Embodiment 2 of this application;
[0031] Figure 3 This is a schematic diagram of a key information detection device for trade documents according to Embodiment 3 of this application;
[0032] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the trade document key information detection method according to the embodiments of this application. Detailed Implementation
[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0034] It should be noted that the terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0035] Example 1
[0036] Figure 1a This is a flowchart of a method for detecting key information in trade documents according to Embodiment 1 of this application. This embodiment is applicable to situations where key fields and other small targets in trade documents are detected. It can be executed by a key information detection device for trade documents, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1a As shown, the method includes:
[0037] S101. Divide the trade document image into multiple image slices;
[0038] S102. Input the image slices into the backbone network of the target detection model, and extract multi-scale feature maps through the backbone network; the multi-scale feature maps include a second layer feature map;
[0039] S103. Input the multi-scale feature map into the neck network of the target detection model, fuse the multi-scale feature map through the neck network to generate an original fused feature map, and perform coordinate attention processing on the original fused feature map to obtain an enhanced feature map.
[0040] S104. Input the enhanced feature map into the detection head of the target detection model, and output the prediction box and the prediction classification result;
[0041] S105. Determine the key information in the trade document image based on the prediction box and the prediction classification result;
[0042] The detection head includes a second detection branch for processing the second layer feature map, and the second detection branch is connected to a feature fusion structure in the neck network that is adapted to the second layer feature map.
[0043] refer to Figure 1bThe object detection model comprises a backbone network, a neck network, and a head. The backbone network includes a second feature extraction layer (P2 layer) for extracting second-layer feature maps. Compared to the third (P3), fourth (P4), and fifth (P5) feature maps in the basic object detection network, the second-layer feature map has a higher resolution, with a size that is one-quarter the size of the image slice in the input object detection model. For example, with an image slice size of 640 pixels, the size of the second-layer feature map is 160 pixels, which improves the detection accuracy of small objects such as key fields. The head includes a second detection branch for processing the second-layer feature map, and this second detection branch is connected to the feature fusion structure in the neck network that adapts the second feature map. The head also includes third, fourth, and fifth detection branches for processing the third, fourth, and fifth-layer features, respectively. The structure of the second detection branch is similar to the other detection branches, containing components such as convolutional layers, batch normalization layers, and activation functions. By adding a second feature extraction layer with higher resolution to the target detection model, the model's ability to detect small targets can be effectively improved, thereby increasing the accuracy of key information detection.
[0044] In this embodiment of the invention, user-uploaded trade document images can be acquired, and preprocessed using algorithms such as spatial domain filtering, channel filtering, or high-contrast overlay to output clearer trade document images. To improve the detection accuracy of small-sized targets such as key fields in the trade document images, the trade document images are segmented into multiple image slices, and these image slices are input into the target detection model. Multi-scale feature maps are extracted from the input image slices through a backbone network. The multi-scale feature maps include second-layer feature maps, third-layer feature maps, fourth-layer feature maps, and fifth-layer feature maps, with the sizes of each feature map being 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image slice, respectively.
[0045] The neck network incorporates a coordinate attention mechanism to process the raw fused feature map output by the C2F module (convolution to feature module). Coordinate attention performs average pooling in both the horizontal and vertical directions to encode spatial location information; and by weighting across channels, it fuses spatial location information with channel information, resulting in an enhanced feature map with both spatial and channel attention. Key fields in trade document images exhibit specific text formats (company name, uppercase and lowercase numbers, date formats, etc.) and their positional distribution within the image follows patterns (appearing at the beginning and end, surrounded by large blank areas, containing seals, etc.). Introducing the coordinate attention mechanism not only improves the object detection model's ability to capture directional features but also enhances the discriminative power of feature representations, making it more suitable for highly directional image data like trade documents and further improving the accuracy of key information detection.
[0046] The enhanced feature maps of each layer output by the neck network are input into the corresponding detection branches in the detection head. Each detection branch includes a regression sub-branch and a classification sub-branch, obtaining the predicted bounding boxes and predicted classification results output by each layer's detection branches; and combining these predicted bounding boxes and predicted classification results, the key information in the trade document image is determined.
[0047] In one alternative implementation, the trade document is a trade contract; the key information includes at least one of the following: the names of the trading parties, the transaction amount, or the transaction date.
[0048] For trade contract detection scenarios, during the target detection model training phase, the actual classification results labeled on the sample data include key fields such as the names of the trading parties, transaction amount, and transaction date. This allows the model to focus only on identifying key fields in trade contract detection scenarios, without needing to identify other fields besides the key fields.
[0049] The technical solution of this embodiment enhances the model's ability to detect small targets by adding a second feature extraction layer with higher resolution to the target detection model, thereby improving the accuracy of key information detection. Furthermore, by segmenting the trade document image into multiple image slices and introducing a coordinate attention mechanism, the model's ability to capture regional directional features is improved, making it more suitable for image data with strong directionality, such as trade documents, and further enhancing the accuracy of key information detection.
[0050] In one optional implementation, segmenting the trade document image into multiple image slices includes: employing a slice-assisted super-inference strategy to segment the trade document image into multiple partially overlapping image slices; determining key information in the trade document image based on the predicted bounding boxes and the predicted classification results includes: using a non-maximum suppression algorithm to filter redundant bounding boxes from the predicted bounding boxes, and determining key information in the trade document image based on the remaining predicted bounding boxes and the corresponding predicted classification results.
[0051] For example, a Slicing-Aided HyperInference (SAHI) strategy is used to partially overlap and segment trade document images, resulting in multiple image slices. Because these slices overlap, the number of predicted bounding boxes is often much larger than the actual number, and these predicted boxes tend to stack on top of each other. Using a non-maximum suppression (NMS) algorithm to filter redundant predicted boxes can improve their accuracy. NMS is a post-processing method in object detection that sorts all predicted boxes according to their confidence level (i.e., class probability) from high to low and selects the box with the highest confidence level as the final predicted box.
[0052] The system acquires the predicted bounding boxes and their confidence scores from the detection head output. A non-maximum suppression algorithm is used to filter redundant boxes, removing those with confidence scores below a threshold (e.g., 0.7) and retaining only those with scores above the threshold. Based on the classification results of the retained boxes, the local coordinates of the slices are mapped back to the original global coordinate system of the trade document image to determine the key information entities and their locations within the image. Since trade document images, such as trade contract images, may be large and have uneven information distribution, a slice-assisted super-inference strategy is used to partially overlap the slices. The detection results of each slice are then merged, effectively addressing the issue of missed detections due to insufficient resolution in full-image detection and further improving the robustness and accuracy of the overall recognition. Given that the key information regions in the trade document image do not overlap, a non-maximum suppression algorithm is used to filter overlapping detection boxes, retaining only the best detection results to ensure high confidence in the final output key information regions.
[0053] Example 2
[0054] Figure 2a This is a flowchart of another method for detecting key information in trade documents according to Embodiment 2 of this application. The technical solution of this embodiment further defines the coordinate attention mechanism based on the above technical solution. See also... Figure 2a The method for detecting key information in trade documents, as shown, includes:
[0055] S201. Divide the trade document image into multiple image slices;
[0056] S202. Input the image slices into the backbone network of the target detection model, and extract multi-scale feature maps through the backbone network; the multi-scale feature maps include a second layer feature map;
[0057] S203. Input the multi-scale feature map into the neck network of the target detection model, and fuse the multi-scale feature map through the neck network to generate an original fused feature map.
[0058] S204. In the horizontal direction, the original fused feature map is processed using the first convolutional kernel to extract horizontal spatial location features and generate horizontal attention weights.
[0059] S205. In the vertical direction, the original fused feature map is processed using a second convolutional kernel to extract vertical spatial location features and generate vertical attention weights.
[0060] S206. Determine the enhanced feature map based on the horizontal attention weight, the vertical attention weight, and the original fused feature map;
[0061] S207. Input the enhanced feature map into the detection head of the target detection model, and output the prediction box and the prediction classification result;
[0062] S208. Based on the prediction box and the prediction classification result, determine the key information in the trade document image;
[0063] The detection head includes a second detection branch for processing the second layer feature map, and the second detection branch is connected to a feature fusion structure in the neck network that is adapted to the second layer feature map; the number of channels of the second convolutional kernel is less than the number of channels of the first convolutional kernel.
[0064] refer to Figure 2bIn the coordinate attention processing, for the horizontal direction, a first convolutional kernel is used to process the original fused feature map, extracting horizontal spatial location features and generating horizontal attention weights. The number of channels in the first convolutional kernel can remain unchanged. For the vertical direction, a second convolutional kernel with a lower number of channels than the first convolutional kernel is used to process the original fused feature map. For example, the number of channels in the second convolutional kernel can be reduced to 1 / r of the original number of channels, or the number of channels in the second convolutional kernel can be directly set to 1, reducing the computational cost of vertical attention. Since the text in the trade document image is arranged horizontally, by keeping the number of channels in the horizontal attention constant, the changes in the feature map in the horizontal direction can be fully captured, thus accurately identifying the horizontally arranged text content. Furthermore, by reducing the number of channels in the vertical convolutional kernel, the interference of the vertical layout structure on key information detection can be reduced, thereby improving the efficiency and accuracy of key information detection.
[0065] In one optional implementation, determining the enhanced feature map based on the lateral attention weight, the vertical attention weight, and the original fusion feature map includes: attenuating the vertical attention weight using an attenuation coefficient to obtain the attenuated vertical attention weight; and performing a weighted fusion of the lateral attention weight, the attenuated vertical attention weight, and the original fusion feature map to obtain the enhanced feature map.
[0066] refer to Figure 2b Following the vertical activation layer is a decay layer. After the vertical activation layer outputs the vertical attention weights, these weights are then input into the decay layer. Referring to the following formula, the decay layer attenuates the vertical attention weights using a decay coefficient, resulting in the attenuated vertical attention weights:
[0067] y = (y0) a
[0068] Where y and y0 are the attenuated vertical attention weights and the original vertical attention weights, respectively, and a is the attenuation coefficient, with a value range of (0,1). Subsequently, the horizontal attention weights output by the horizontal activation layer, the attenuated vertical attention weights, and the original fused feature map are weighted and fused to obtain the enhanced feature map. By using an attenuation coefficient to attenuate the vertical attention weights, the impact of the vertical layout structure on key information detection can be further reduced.
[0069] The technical solution of this embodiment, by keeping the number of channels for lateral attention constant, can fully capture the changes in horizontally arranged text in trade document images, thereby accurately identifying the horizontal text content. Furthermore, by reducing the number of channels in the vertical convolution kernel and using an attenuation coefficient to attenuate the vertical attention weights, the interference of the vertical layout structure on key information detection can be reduced, improving the efficiency and accuracy of key information detection.
[0070] In one optional implementation, the loss function of the regression sub-branch in the detection head is determined as follows: calculate the intersection-union ratio (IU) between the predicted box and the ground truth box; calculate the center point distance between the center point of the predicted box and the center point of the ground truth box, and the diagonal length of the smallest bounding box that includes both the predicted box and the ground truth box; and determine the loss function of the regression sub-branch based on the IU, the center point distance, and the diagonal length.
[0071] During the training phase, the object detection model can be initially trained on a large general object detection dataset to improve its feature extraction capabilities. Then, it can be transferred to a dataset of trade document images for fine-tuning to obtain an object detection model for trade document images.
[0072] In the scenario of trade document detection, the regression sub-branch uses CIoU (Complete Intersection over Union) to calculate the loss. In addition to IoU (Intersection over Union), it also considers the distance between the center points and the aspect ratio, which can improve the accuracy of target detection. The classification sub-branch uses the cross-entropy loss function instead of the binary cross-entropy to calculate the difference between the classification probability predicted by the model and the true label distribution.
[0073] For example, the specific calculation process of the loss function of the regression sub-branch is as follows: calculate the intersection-over-union ratio (IoU) between the predicted box and the ground truth box; calculate the Euclidean distance between the center point bp of the predicted box and the center point bt of the ground truth box, denoted as ρ. 2 (bp, bt); and determine the diagonal length c of the smallest bounding box that can simultaneously contain both the predicted and ground truth boxes, which reflects aspect ratio information; based on the calculated intersection-over-union (IoU) and center point distance ρ 2 Given (bp, bt) and the minimum bounding box diagonal length c (which involves aspect ratio), calculate the loss function L of the regression sub-branch according to the following formula. cIOU :
[0074]
[0075] Where v is used to measure the similarity of the aspect ratio between the predicted box and the ground truth box, and α is used as a weighting coefficient to balance the influence of aspect ratio similarity.
[0076] Example 3
[0077] Figure 3This is a schematic diagram of a key information detection device for trade documents according to Embodiment 3 of this application. This embodiment is applicable to situations where key fields and other small targets in trade documents are detected. This key information detection device for trade documents can be implemented in hardware and / or software, and can be configured in an electronic device. (Reference) Figure 3 The specific structure of the key information detection device 300 for trade documents is as follows:
[0078] Image slicing module 310 is used to segment trade document images into multiple image slices;
[0079] Feature extraction module 320 is used to input the image slices into the backbone network of the target detection model, and extract multi-scale feature maps through the backbone network; the multi-scale feature maps include a second layer feature map;
[0080] The feature fusion module 330 is used to input the multi-scale feature map into the neck network of the target detection model, fuse the multi-scale feature map through the neck network to generate an original fused feature map, and perform coordinate attention processing on the original fused feature map to obtain an enhanced feature map.
[0081] The prediction output module 340 is used to input the enhanced feature map into the detection head of the target detection model and output the prediction box and the prediction classification result;
[0082] The key information determination module 350 is used to determine key information in the trade document image based on the prediction box and the prediction classification result;
[0083] The detection head includes a second detection branch for processing the second layer feature map, and the second detection branch is connected to a feature fusion structure in the neck network that is adapted to the second layer feature map.
[0084] In one optional implementation, the feature fusion module 330 includes:
[0085] The lateral processing unit is used to process the original fused feature map in the lateral direction using a first convolutional kernel to extract lateral spatial location features and generate lateral attention weights.
[0086] The vertical processing unit is used to process the original fused feature map in the vertical direction using a second convolutional kernel to extract vertical spatial location features and generate vertical attention weights.
[0087] An enhanced feature unit is used to determine an enhanced feature map based on the lateral attention weights, the vertical attention weights, and the original fused feature map.
[0088] The second convolutional kernel has fewer channels than the first convolutional kernel.
[0089] In one alternative implementation, the enhanced feature unit is specifically used for:
[0090] The longitudinal attention weights are attenuated using an attenuation coefficient to obtain the attenuated longitudinal attention weights;
[0091] The enhanced feature map is obtained by weighting and fusing the horizontal attention weights, the attenuated vertical attention weights, and the original fused feature map.
[0092] In one optional implementation, the image slicing module 310 is specifically used to: employ a slice-assisted super-inference strategy to segment the trade document image into multiple partially overlapping image slices;
[0093] The key information determination module 350 is specifically used for:
[0094] The nonmaximum suppression algorithm is used to filter redundant boxes in the predicted bounding boxes, and the key information in the trade document image is determined based on the remaining predicted bounding boxes and the corresponding prediction classification results.
[0095] In one optional implementation, the loss function of the regression sub-branch in the detection head is determined as follows:
[0096] Calculate the intersection-union ratio (IU) between the predicted bounding boxes and the ground truth bounding boxes;
[0097] Calculate the center distance between the center point of the predicted box and the center point of the ground truth box, and the diagonal length of the smallest bounding box that includes both the predicted box and the ground truth box.
[0098] The loss function of the regression sub-branch is determined based on the intersection-union ratio, the distance between the center points, and the diagonal length.
[0099] In one alternative implementation, the trade document is a trade contract; the key information includes at least one of the following: the names of the trading parties, the transaction amount, or the transaction date.
[0100] This application provides an enhanced detection capability for small target text regions in trade document images by introducing a second layer of feature processing. Given the significant directional correlation of trade document text, a coordinate attention module is introduced to more accurately capture the direction and position information of text regions. Furthermore, a slice-assisted super-inference strategy is employed to slice the trade document image to enhance detection performance, further improving the detection capability of local information in complex or large-sized documents, and the detection results of each slice are merged. Moreover, a non-maximum suppression algorithm is used to filter and retain the best detection results, thereby accurately identifying and displaying key information in the trade document file.
[0101] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.
[0102] Example 4
[0103] Figure 4 This is a schematic diagram of the structure of an electronic device 410 implementing the trade document key information detection method according to embodiments of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.
[0104] like Figure 4 As shown, the electronic device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.
[0105] Multiple components in electronic device 410 are connected to I / O interface 415, including: input unit 416, such as keyboard, mouse, etc.; output unit 417, such as various types of displays, speakers, etc.; storage unit 418, such as disk, optical disk, etc.; and communication unit 419, such as network card, modem, wireless transceiver, etc. Communication unit 419 allows electronic device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0106] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as the method for detecting key information in trade documents.
[0107] In some embodiments, the trade document key information detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the trade document key information detection method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured as the trade document key information detection method by any other suitable means (e.g., by means of firmware).
[0108] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0109] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0110] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0111] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0112] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0113] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0114] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.
[0115] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method for detecting key information in trade documents, characterized in that, include: The trade document image is segmented into multiple image slices; The image slices are input into the backbone network of the target detection model, and multi-scale feature maps are extracted through the backbone network. The multi-scale feature map includes a second-layer feature map; The multi-scale feature map is input into the neck network of the target detection model. The neck network fuses the multi-scale feature map to generate an original fused feature map. The original fused feature map is then subjected to coordinate attention processing to obtain an enhanced feature map. The enhanced feature map is input into the detection head of the target detection model, and the predicted bounding box and the predicted classification result are output. Based on the prediction box and the prediction classification result, determine the key information in the trade document image; The detection head includes a second detection branch for processing the second layer feature map, and the second detection branch is connected to a feature fusion structure in the neck network that is adapted to the second layer feature map.
2. The method according to claim 1, characterized in that, The original fused feature map is subjected to coordinate attention processing to obtain an enhanced feature map, including: In the horizontal direction, the original fused feature map is processed using the first convolutional kernel to extract horizontal spatial location features and generate horizontal attention weights; In the vertical direction, the original fused feature map is processed using a second convolutional kernel to extract vertical spatial location features and generate vertical attention weights; The enhanced feature map is determined based on the lateral attention weight, the vertical attention weight, and the original fused feature map; The second convolutional kernel has fewer channels than the first convolutional kernel.
3. The method according to claim 2, characterized in that, The step of determining the enhanced feature map based on the lateral attention weight, the vertical attention weight, and the original fused feature map includes: The longitudinal attention weights are attenuated using an attenuation coefficient to obtain the attenuated longitudinal attention weights; The enhanced feature map is obtained by weighting and fusing the horizontal attention weights, the attenuated vertical attention weights, and the original fused feature map.
4. The method according to claim 1, characterized in that, The step of segmenting the trade document image into multiple image slices includes: using a slice-assisted super-inference strategy to segment the trade document image into multiple partially overlapping image slices; The step of determining key information in the trade document image based on the prediction box and the prediction classification result includes: The nonmaximum suppression algorithm is used to filter redundant boxes in the predicted bounding boxes, and the key information in the trade document image is determined based on the remaining predicted bounding boxes and the corresponding prediction classification results.
5. The method according to claim 1, characterized in that, The loss function of the regression sub-branch in the detection head is determined as follows: Calculate the intersection-union ratio (IU) between the predicted bounding boxes and the ground truth bounding boxes; Calculate the center distance between the center point of the predicted box and the center point of the ground truth box, and the diagonal length of the smallest bounding box that includes both the predicted box and the ground truth box. The loss function of the regression sub-branch is determined based on the intersection-union ratio, the distance between the center points, and the diagonal length.
6. The method according to claim 1, characterized in that, The trade document is a trade contract; the key information includes at least one of the following: the names of the trading parties, the transaction amount, or the transaction date.
7. A device for detecting key information in trade documents, characterized in that, include: The image slicing module is used to segment trade document images into multiple image slices; The feature extraction module is used to input the image slices into the backbone network of the target detection model, and extract multi-scale feature maps through the backbone network; the multi-scale feature maps include a second layer feature map; The feature fusion module is used to input the multi-scale feature map into the neck network of the target detection model, fuse the multi-scale feature map through the neck network to generate an original fused feature map, and perform coordinate attention processing on the original fused feature map to obtain an enhanced feature map. The prediction output module is used to input the enhanced feature map into the detection head of the target detection model and output the prediction box and the prediction classification result; The key information determination module is used to determine key information in the trade document image based on the prediction box and the prediction classification result; The detection head includes a second detection branch for processing the second layer feature map, and the second detection branch is connected to a feature fusion structure in the neck network that is adapted to the second layer feature map.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the trade document key information detection method as described in any one of claims 1-6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the trade document key information detection method as described in any one of claims 1-6.
10. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the trade document key information detection method as described in any one of claims 1-6.