Bill image classification method and device

Abstract

The invention discloses a bill image classification method and device, and relates to the field of financial science and technology or other related fields, and the classification method comprises the steps: carrying out the preprocessing of a bill image, obtaining a target bill image, processing the target bill image through a preset feature processing model, obtaining a target weight feature map, and obtaining a target weight feature map; and adopting a preset classifier to classify the target weight feature map to obtain an image category of the bill image. According to the invention, the technical problem of low classification accuracy of the bill image in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to a method and device for classifying bill images. Background Art

[0002] With the digital transformation of financial services, the demand for electronic bill processing is increasing. Current bill processing methods rely on manual input and review, which is inefficient and prone to errors. Therefore, automated bill classification has become a pressing issue in current financial services.

[0003] In related technologies, the following methods are mainly used for bill classification: (1) Conventional image processing method: image templates are made according to different bill standard images, and then image classification is performed by comparing similarities; (2) Image classification method based on deep learning: convolutional neural network is used for training, and texture features and semantic features between images are learned through convolution calculation and pooling operation; (3) A deeper network model is used to learn the detailed information of the image through larger parameters.

[0004] However, the above-mentioned bill classification has the following problems: (1) Conventional image processing methods have difficulty in feature selection, and manually extracted features may not be sufficient to express effective image information; (2) Image classification methods based on deep learning can only extract local features of images, but have limited processing capabilities for multi-scale information. Bill images usually contain different types of patterns, text, and background information, which require comprehensive analysis at different scales. However, this method ignores the global information and important areas in the image, resulting in the classification results being affected by local features and low accuracy; (3) Excessive network depth will lead to a large increase in parameters, and the model's generalization ability is not strong, which can easily lead to large deviations in actual application and test results.

[0005] Therefore, the bill classification method in the related art has problems such as low efficiency and poor accuracy, especially when faced with bills of various types and complex formats.

[0006] To the above problems, no effective solution has been proposed yet. Summary of the Invention

[0007] The embodiments of the present invention provide a bill image classification method and apparatus thereof, so as to at least solve the technical problem of low accuracy in classifying bill images in the related art.

[0008] According to one aspect of an embodiment of the present invention, a method for classifying bill images is provided, including: preprocessing the bill image to obtain a target bill image; processing the target bill image using a preset feature processing model to obtain a target weight feature map, wherein the preset feature processing model at least includes: a multi-scale attention feature fusion structure; the multi-scale attention feature fusion structure is used to extract feature maps of the target bill image at different scales, and weight the feature maps at different scales to obtain a target weight feature map; and classifying the target weight feature map using a preset classifier to obtain an image category of the bill image.

[0009] Furthermore, the step of pre-processing the bill image to obtain the target bill image includes: performing grayscale processing on the bill image to obtain a grayscale image; and normalizing the grayscale image to obtain the target bill image.

[0010] Furthermore, before using the preset feature processing model to process the target bill image and obtain the target weight feature map, it also includes: constructing a multi-scale attention feature fusion structure, wherein the multi-scale attention feature fusion structure includes at least: multiple layers of convolutional layers, an attention structure connected to each convolutional layer, and a fusion structure; based on the multi-scale attention feature fusion structure, constructing an initial feature processing model; combining with the preset classifier, training the initial feature processing model to obtain the preset feature processing model.

[0011] Furthermore, in combination with a preset classifier, the initial feature processing model is trained to obtain the preset feature processing model, including the following steps: obtaining a set of historical bill images; performing expansion processing on each historical bill image in the historical bill image set to obtain a training set, wherein the expansion processing includes at least one of the following: rotation, scaling, and flipping; each training sample in the training set is associated with a labeled category; each training sample is a historical bill image or an expanded historical bill image; in combination with the preset classifier, the initial feature processing model is iterated using the training set to obtain the preset feature processing model, wherein the process of iterating the initial feature processing model using the training set includes: inputting each training sample into the initial feature processing model in turn to obtain a prediction weight feature map; inputting each prediction weight feature map into the preset classifier in turn to obtain a prediction category; adjusting the model parameters in the initial feature processing model based on each prediction category and the labeled category corresponding to each prediction category until the loss value determined based on the prediction category and the labeled category is less than the preset loss threshold.

[0012] Furthermore, the multi-scale attention feature fusion structure includes at least: a first convolutional layer, a second convolutional layer, and a third convolutional layer, and uses a preset feature processing model to process the target bill image to obtain a target weight feature map, including: using the first convolutional layer to extract features from the target bill image to obtain a first feature map; using the second convolutional layer to extract features from the first feature map to obtain a second feature map, wherein the scale of the second feature map is smaller than the scale of the first feature map; using the third convolutional layer to extract features from the second feature map to obtain a third feature map, wherein the scale of the third feature map is smaller than the scale of the second feature map; using the spatial attention structure to process the first feature map to obtain a spatial weight feature map; using the spatial attention structure and the channel attention structure to process the second feature map to obtain a fused weight feature map; using the channel attention structure to process the third feature map to obtain a channel weight feature map; using the fusion structure in the multi-scale attention feature fusion structure to fuse the spatial weight feature map, the fused weight feature map and the channel weight feature map to obtain a target weight feature map.

[0013] Furthermore, the step of using the spatial attention structure to process the first feature map to obtain a spatial weight feature map includes: performing maximum pooling operations, convolution operations, and average pooling operations on the first feature map to obtain a first sub-feature map, a second sub-feature map, and a third sub-feature map; splicing the first sub-feature map, the second sub-feature map, and the third sub-feature map according to the channel dimension to obtain a multi-channel feature map; convolving the multi-channel feature map to obtain a first single-channel feature map, and processing the first single-channel feature map with an activation function to obtain a first spatial attention map; and performing dot multiplication of the first spatial attention map and the first feature map to obtain a spatial weight feature map.

[0014] Furthermore, the step of using the spatial attention structure and the channel attention structure to process the second feature map to obtain a fused weight feature map includes: copying the second feature map to obtain a second feature copy map; processing the second feature map using the spatial attention structure to obtain a second spatial attention map, and performing dot product of the second spatial attention map and the second feature map to obtain a first weight feature map; processing the second feature copy map using the channel attention structure to obtain a first channel attention map, and performing dot product of the first channel attention map and the second feature copy map to obtain a second weight feature map; fusing the first weight feature map and the second weight feature map to obtain a fused weight feature map.

[0015] Furthermore, the step of using the channel attention structure to process the third feature map to obtain a channel weight feature map includes: performing maximum pooling operations and average pooling operations on the third feature map to obtain a fourth sub-feature map and a fifth sub-feature map; convolving the fourth sub-feature map to obtain a first single-channel map, and convolving the fifth sub-feature map to obtain a second single-channel map; splicing the first single-channel map and the second single-channel map to obtain a spliced single-channel map; convolving the spliced single-channel map to obtain a second single-channel feature map, and processing the second single-channel feature map with an activation function to obtain a second channel attention map; and performing dot multiplication on the second channel attention map and the third feature map to obtain a channel weight feature map.

[0016] Furthermore, the fusion structure in the multi-scale attention feature fusion structure is used to fuse the spatial weight feature map, the fusion weight feature map and the channel weight feature map to obtain the target weight feature map, including: downsampling the spatial weight feature map to obtain a downsampled spatial weight feature map with the same scale as the fusion weight feature map; upsampling the channel weight feature map to obtain an upsampled channel weight feature map with the same scale as the fusion weight feature map; and using the fusion structure to fuse the downsampled spatial weight feature map, the fusion weight feature map and the upsampled channel weight feature map to obtain the target weight feature map.

[0017] According to another aspect of an embodiment of the present invention, a device for classifying bill images is also provided, including: a first processing unit, used to preprocess the bill image to obtain a target bill image; a second processing unit, used to process the target bill image using a preset feature processing model to obtain a target weight feature map, wherein the preset feature processing model at least includes: a multi-scale attention feature fusion structure; the multi-scale attention feature fusion structure is used to extract feature maps of the target bill image at different scales, and weight the feature maps at different scales to obtain a target weight feature map; a classification unit, used to classify the target weight feature map using a preset classifier to obtain the image category of the bill image.

[0018] Furthermore, the first processing unit includes: performing grayscale processing on the bill image to obtain a grayscale image; and normalizing the grayscale image to obtain a target bill image.

[0019] Furthermore, the classification device also includes: a first construction unit, used to construct a multi-scale attention feature fusion structure before using a preset feature processing model to process the target bill image and obtain a target weight feature map, wherein the multi-scale attention feature fusion structure includes at least: multiple layers of convolutional layers, an attention structure connected to each convolutional layer, and a fusion structure; a second construction unit, used to construct an initial feature processing model based on the multi-scale attention feature fusion structure; a first training unit, used to train the initial feature processing model in combination with a preset classifier to obtain a preset feature processing model.

[0020] Furthermore, the first training unit includes: a first acquisition module for acquiring a set of historical bill images; a first expansion module for performing expansion processing on each historical bill image in the set of historical bill images to obtain a training set, wherein the expansion processing includes at least one of the following: rotation, scaling, and flipping; each training sample in the training set is associated with a labeled category; each training sample is a historical bill image or an expanded historical bill image; a first iteration module for combining a preset classifier and using the training set to iterate the initial feature processing model to obtain a preset feature processing model, wherein the process of iterating the initial feature processing model using the training set includes: inputting each training sample into the initial feature processing model in turn to obtain a prediction weight feature map; inputting each prediction weight feature map into the preset classifier in turn to obtain a prediction category; based on each prediction category and the labeled category corresponding to each prediction category, adjusting the model parameters in the initial feature processing model until the loss value determined based on the prediction category and the labeled category is less than the preset loss threshold.

[0021] Furthermore, the multi-scale attention feature fusion structure includes at least: a first convolutional layer, a second convolutional layer, and a third convolutional layer, and the second processing unit includes: a first extraction module, which is used to use the first convolutional layer to perform feature extraction on the target bill image to obtain a first feature map; a second extraction module, which is used to use the second convolutional layer to perform feature extraction on the first feature map to obtain a second feature map, wherein the scale of the second feature map is smaller than the scale of the first feature map; a third extraction module, which is used to use the third convolutional layer to perform feature extraction on the second feature map to obtain a third feature map, wherein the scale of the third feature map is smaller than the scale of the second feature map; a first processing module, which is used to use the spatial attention structure to process the first feature map to obtain a spatial weight feature map; a second processing module, which is used to use the spatial attention structure and the channel attention structure to process the second feature map to obtain a fused weight feature map; a third processing module, which is used to use the channel attention structure to process the third feature map to obtain a channel weight feature map; a first fusion module, which is used to use the fusion structure in the multi-scale attention feature fusion structure to fuse the spatial weight feature map, the fused weight feature map and the channel weight feature map to obtain a target weight feature map.

[0022] Furthermore, the first processing module includes: a first operation submodule, which is used to perform maximum pooling operation, convolution operation and average pooling operation on the first feature map, respectively, to obtain a first sub-feature map, a second sub-feature map and a third sub-feature map; a first splicing submodule, which is used to splice the first sub-feature map, the second sub-feature map and the third sub-feature map according to the channel dimension to obtain a multi-channel feature map; a first convolution submodule, which is used to convolve the multi-channel feature map to obtain a first single-channel feature map, and process the first single-channel feature map using an activation function to obtain a first spatial attention map; a first point multiplication submodule, which is used to perform point multiplication on the first spatial attention map and the first feature map to obtain a spatial weight feature map.

[0023] Furthermore, the second processing module includes: a first copying submodule, used to copy the second feature map to obtain a second feature copy map; a first processing submodule, used to process the second feature map using a spatial attention structure to obtain a second spatial attention map, and perform dot multiplication of the second spatial attention map with the second feature map to obtain a first weight feature map; a second processing submodule, used to process the second feature copy map using a channel attention structure to obtain a first channel attention map, and perform dot multiplication of the first channel attention map with the second feature copy map to obtain a second weight feature map; a third processing submodule, used to perform fusion processing on the first weight feature map and the second weight feature map to obtain a fused weight feature map.

[0024] Furthermore, the third processing module includes: a second operation submodule, which is used to perform maximum pooling operations and average pooling operations on the third feature map, respectively, to obtain a fourth sub-feature map and a fifth sub-feature map; a second convolution submodule, which is used to convolve the fourth sub-feature map to obtain a first single-channel map, and to convolve the fifth sub-feature map to obtain a second single-channel map; a second splicing submodule, which is used to splice the first single-channel map and the second single-channel map to obtain a spliced single-channel map; a third convolution submodule, which is used to convolve the spliced single-channel map to obtain a second single-channel feature map, and process the second single-channel feature map using an activation function to obtain a second channel attention map; a second point multiplication submodule, which is used to perform point multiplication on the second channel attention map and the third feature map to obtain a channel weight feature map.

[0025] Furthermore, the first fusion module includes: a first downsampling submodule, used to downsample the spatial weight feature map to obtain a downsampled spatial weight feature map with the same scale as the fusion weight feature map; a first upsampling submodule, used to upsample the channel weight feature map to obtain an upsampled channel weight feature map with the same scale as the fusion weight feature map; a first fusion submodule, used to fuse the downsampled spatial weight feature map, the fusion weight feature map and the upsampled channel weight feature map using a fusion structure to obtain a target weight feature map.

[0026] According to another aspect of an embodiment of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned bill image classification methods is implemented.

[0027] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any of the above-mentioned bill image classification methods.

[0028] In the present invention, the bill image is preprocessed to obtain a target bill image, the target bill image is processed using a preset feature processing model to obtain a target weight feature map, and the target weight feature map is classified using a preset classifier to obtain the image category of the bill image, thereby solving the technical problem of low accuracy in classifying bill images in related technologies.

[0029] In the present invention, by adding a multi-scale attention feature fusion structure to the preset feature processing model, the feature maps of the bill image at different scales can be extracted, and the feature maps at different scales are weighted, so that the target weight feature map of the fusion weight can be obtained. After that, the target weight feature map is classified by the preset classifier, and the image category of the bill image can be accurately obtained, thereby achieving the technical effect of improving the accuracy and robustness of bill image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0031] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a bill image classification method is shown;

[0032] Figure 2is a flow chart of a bill image classification method according to embodiment 1 of the present invention;

[0033] Figure 3 2. It is a schematic diagram of a multi-scale attention feature fusion structure according to embodiment 1 of the present invention;

[0034] Figure 4 2 is a schematic diagram of an attention-based multi-scale feature fusion framework according to embodiment 1 of the present invention;

[0035] Figure 5 2 is a schematic diagram of the processing flow of the spatial attention module according to embodiment 1 of the present invention;

[0036] Figure 6 2 is a schematic diagram of a processing flow of a channel attention module according to embodiment 1 of the present invention;

[0037] Figure 7 is a schematic diagram of an optional bill image classification device according to an embodiment of the present invention;

[0038] Figure 8 is a structural block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] To facilitate those skilled in the art to understand the present invention, some of the terms or nouns involved in the embodiments of the present invention are explained below:

[0042] Bill image: refers to the electronic image of various types of financial bills (such as checks, bills of exchange, receipts, etc.).

[0043] Image classification: refers to assigning input images to predefined categories through computer vision techniques.

[0044] Multi-scale attention mechanism: It is a technique in deep learning models that aims to capture important features of each part of the image through contextual information at different scales.

[0045] Convolutional Neural Network (CNN): A deep learning model suitable for image data. It extracts image features through multiple layers of convolutional layers, pooling layers, and fully connected layers. It is often used for image classification tasks.

[0046] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected and involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set up between this system and the relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0047] In the present invention, the current image classification method uses the attention mechanism to learn only the feature maps at the same level, while ignoring the different information concerned by the feature maps at different levels. However, in the process of convolution calculation of feature maps, the image information expressed by the feature maps at different levels has different characteristics. The low-level convolution has a small receptive field, a large feature map, rich spatial information, but insufficient semantic information; the high-level convolution has a large receptive field, a small feature map, strong semantic information representation ability, but lacks spatial information. A bill image classification method based on a multi-scale attention mechanism is proposed. By combining the convolutional neural network and the attention mechanism in deep learning, the key features in the bill image can be effectively extracted, and the classification accuracy and robustness can be improved.

[0048] In the present invention, by introducing multi-scale contextual information and an adaptive attention focusing mechanism, the problem of ignoring global features and local key areas in current image classification methods is solved, thereby improving the accuracy and robustness of bill image classification.

[0049] Since the feature maps become smaller and smaller and the semantic information expressed becomes richer and richer during the process of convolution calculation of feature maps, it is necessary to express feature information of different levels on feature maps of different levels and adjust the attention focus of the model according to the importance. In the present invention, by introducing a multi-scale attention mechanism, it is possible to learn image information at multiple scales, and to enhance the weights of important features through the spatial attention module (SpatialAttention Module, SAM) and the channel attention module (ChannelAttention Module, CAM) to effectively cope with the diversity and complexity of bill images, thereby accurately capturing the key features in the bill images and improving the accuracy of classification. Moreover, by fusing information at different scales, it is possible to solve the challenges brought about by the differences in format, size, font, etc. of bill images.

[0050] The present invention will be described in detail below with reference to various embodiments.

[0051] Example 1

[0052] According to an embodiment of the present application, an embodiment of a method for classifying bill images is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0053] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a bill image classification method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more ( Figure 1 102a, 102b, ..., 102n are used to illustrate) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / 0 interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera, wherein the network interface may be connected to a wired and / or wireless network. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0054] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0055] The memory 104 can be used to store software programs and modules for application software, such as the program instructions / data storage device corresponding to the bill image classification method in the embodiment of the present application. The processor 102 executes the software programs and modules stored in the memory 104 to perform various functional applications and data processing, thereby implementing the above-mentioned bill image classification method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, and such remote memory may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0056] The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0057] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0058] Under the above operating environment, this application provides Figure 2 The classification method of the bill image shown. Figure 2 : is a flow chart of the bill image classification method according to embodiment 1 of the present invention, as shown in FIG. Figure 2 As shown, the method includes the following steps:

[0059] Step S201: pre-process the bill image to obtain a target bill image.

[0060] In the embodiment of the present invention, the bill image includes elements such as text information, numbers, pictures, and watermarks. By preprocessing the bill image, the original bill image can be converted into a format suitable for model input, ensuring the consistency of the image data input to the model.

[0061] In the embodiment of the present invention, pre-processing of the bill image includes but is not limited to: grayscale conversion, normalization, size standardization, etc.

[0062] Step S202: Use a preset feature processing model to process the target bill image to obtain a target weight feature map, wherein the preset feature processing model includes at least: a multi-scale attention feature fusion structure; the multi-scale attention feature fusion structure is used to extract feature maps of the target bill image at different scales, and weight the feature maps at different scales to obtain a target weight feature map.

[0063] In an embodiment of the present invention, for the preprocessed target bill image, a preset feature processing model, i.e., a model including a multi-scale attention feature fusion structure, can be used to perform feature extraction to obtain a multi-scale and weighted target weight feature map.

[0064] In an embodiment of the present invention, the multi-scale attention feature fusion structure can extract feature maps of the target bill image at different scales, and assign weights to the feature maps at different scales to obtain a target weighted feature map.

[0065] For example, the multi-scale attention feature fusion structure extracts low-, medium-, and high-scale image features through convolutional layers of different depths (e.g., Conv-2, Conv-5, and Conv-8), respectively. Low-level features (such as those extracted by Conv-2) primarily contain local information such as texture and edges, while high-level features (such as those extracted by Conv-8) contain more semantic information. Furthermore, to fuse feature maps of different scales, the feature maps can be resized through upsampling or downsampling to match them. For example, the Conv-2 feature map is reduced to the same size as the Conv-5 feature map through downsampling operations (such as ROIpooling (Region of Interest Pooling)), while the Conv-8 feature map is increased to the same size through upsampling (such as nearest neighbor interpolation). The resized feature maps are then fused, perhaps by concatenating them and then integrating them through 1x1 convolution to preserve multi-scale information. After that, the fused feature map can be weighted by the spatial attention module and channel attention module in the multi-scale attention feature fusion structure to obtain the target weight feature map. Among them, the spatial attention module can extract the spatial information of different regions through convolution and pooling operations, and use activation functions (such as Sigmoid) to generate a spatial weight map, which emphasizes the spatial information of key areas in the bill image (such as the signature area and the amount area) and ignores the background or irrelevant areas. The channel attention module can learn the semantic information of different channels, calculate the weight of each channel through the fully connected layer and activation function, highlight the channels with classification value features, and reduce the interference of irrelevant features.

[0066] Step S203: Using a preset classifier to classify the target weight feature map to obtain the image category of the bill image.

[0067] In an embodiment of the present invention, a preset classifier (which may include one or more fully connected layers and a Softmax activation function to output image category labels, such as 0, 1, 2, etc., representing check, bill of exchange, and receipt, respectively) can be used to classify the feature map weighted by the attention mechanism (i.e., the target weight feature map). The classifier maps the information in the target weight feature map to the category of the bill, such as different image categories such as "check," "bill of exchange," or "receipt."

[0068] Exemplarily, the target weight feature map is a vector, each element of which corresponds to a category. These elements can be converted into probability distributions through the Softmax function in the classifier, and the category with the highest probability is finally selected as the image category of the bill image.

[0069] In summary, by adding a multi-scale attention feature fusion structure to the preset feature processing model, the feature maps of the bill image at different scales can be extracted, and the feature maps at different scales can be weighted to obtain the target weight feature map of the fusion weight. After that, the target weight feature map is classified by the preset classifier, and the image category of the bill image can be accurately obtained, thereby achieving the technical effect of improving the accuracy and robustness of the bill image classification, and then solving the technical problem of low accuracy in classifying bill images in related technologies.

[0070] In order to accurately pre-process the bill image, in the bill image classification method provided in Example 1 of the present application, the bill image is gray-scale processed to obtain a gray-scale image; and the gray-scale image is normalized to obtain a target bill image.

[0071] In an embodiment of the present invention, the original bill image can be preprocessed to ensure that the model can efficiently and accurately extract features and classify them. The preprocessing includes grayscale processing and normalization, as follows:

[0072] The input full-color ticket image can be converted to a grayscale image. For example, this can be achieved by converting the image's RGB (Red, Green, Blue) channel values to grayscale values. This simplifies the rich color information of the ticket image into a single grayscale value, reducing the data dimension and computational complexity. The grayscale processed image not only retains the clarity and contrast of the ticket content but also removes unnecessary color information, allowing the model to focus more on features such as texture, shape, and structure.

[0073] Next, the grayscale image is normalized to further optimize the format of the input data. Normalization adjusts pixel values to a fixed range, such as scaling each pixel value from 0 to 255 to 0 to 1. This ensures that all pixel values in the input image are within the same range, preventing the adverse effects of numerical range differences on model training and improving the model's convergence speed and accuracy.

[0074] In this embodiment, grayscale processing reduces image processing complexity and computing resource consumption while preserving key information in the bill image, such as the boundaries and shapes of text, numbers, and patterns. Normalization standardizes input data, ensuring that data from different bill images has equal weight in the model, thus avoiding training instability and inaccurate classification caused by differences in data range.

[0075] In order to accurately construct a preset feature processing model, in the bill image classification method provided in Example 1 of the present application, before using the preset feature processing model to process the target bill image and obtain the target weight feature map, a multi-scale attention feature fusion structure is constructed, wherein the multi-scale attention feature fusion structure includes at least: multiple layers of convolutional layers, an attention structure connected to each convolutional layer, and a fusion structure; based on the multi-scale attention feature fusion structure, an initial feature processing model is constructed; combined with the preset classifier, the initial feature processing model is trained to obtain the preset feature processing model.

[0076] In an embodiment of the present invention, in order to achieve high accuracy and robustness in bill image classification, a multi-scale attention feature fusion structure is constructed, and based on this, a feature processing model is created and trained.

[0077] Specifically, we first constructed multiple convolutional layers (such as Conv-2, Conv-5, and Conv-8) to capture the features of bill images at different scales. Each convolutional layer has different kernel sizes, numbers, and strides to accommodate the diversity and complexity of bill images. Lower convolutional layers (such as Conv-2) focus on extracting low-level features such as edges and textures, while deeper convolutional layers (such as Conv-8) are more inclined to extract high-level features such as semantics and structure.

[0078] After each convolutional layer, an adaptive attention structure is connected, including the spatial attention module (SAM) and the channel attention module (CAM). The spatial attention module helps the model focus on key areas of the bill (such as the signature area and the amount area) by learning weights for different positions, while the channel attention module enhances the model's capture of important semantic features by learning weights for different feature channels. For example, for the Conv-5 layer, the spatial weights and channel weights calculated by SAM and CAM, respectively, are applied to the feature map to highlight the most valuable information in that layer. In addition, a fusion structure is constructed to fuse the convolutional features of different layers.

[0079] Then, based on the multi-scale attention feature fusion structure, an initial feature processing model was constructed. The front end of the model is a multi-scale attention feature extraction module, and the back end is connected to a classifier to map the extracted features to specific bill categories.

[0080] Next, the initial feature processing model is trained using a large dataset of labeled bill images, combined with a pre-set classifier. During training, the model continuously adjusts its parameters, such as weights and biases, through a backpropagation algorithm to minimize classification error. Through a multi-scale attention feature fusion architecture, the model learns the features and weights of bill images at different scales, resulting in a highly accurate "pre-set feature processing model" upon completion of training.

[0081] Here, the pre-set classifier includes a fully connected layer and a softmax layer, which converts the fused feature map into a probability distribution, thereby determining the most likely category for each bill image. During the model training phase, the classifier output is compared with the true label to guide the adjustment of model parameters and improve classification accuracy.

[0082] In this embodiment, by constructing and training a pre-set feature processing model based on a multi-scale attention feature fusion structure, the accuracy and efficiency of bill image classification can be improved. Specifically, the combination of multiple convolutional layers and the attention structure ensures that the model can capture key multi-scale features from bill images, while the fusion structure integrates these features to form a comprehensive and high-density feature representation.

[0083] Figure 3 is a schematic diagram of the multi-scale attention feature fusion structure according to embodiment 1 of the present invention, as shown Figure 3 As shown, it includes: the first convolution layer (such as Conv-2), the second convolution layer (such as Conv-5), the third convolution layer (such as Conv-8), 1x1 convolution layer (such as Conv 1x1, 64 (i.e., the convolution kernel is 1x1 and the number of channels is 64)) and other multi-layer convolutions, downsampling modules (DownSampling), upsampling modules (UpSampling), fusion structures (Concat), etc., wherein the feature map extracted by Conv-2 is converted into a feature map of the same size as the Conv-5 layer through DownSampling, and the feature map extracted by Conv-8 is converted into a feature map of the same size as the Conv-5 layer through UpSampling, and then the feature maps of different convolution layers are fused through Concat, and convolution is performed through Conv 1x1, 64 to obtain the final feature map.

[0084] exist Figure 3 In the image processing layer, Conv-2 is used to extract low-level features such as edges and textures, Conv-5 is used to extract mid-level features, and Conv-8 is used to extract high-level features. Each convolutional layer can be connected to a pooling layer to reduce the dimensionality of the data and extract the main features of the data.

[0085] exist Figure 3In

[15] , feature maps of different scales are used for feature fusion. For the low-level convolution Conv-2, the number of pooling layers is small, the feature map is large, and it has high texture information. To fuse with the deep feature map, downsampling is required to reduce the size. For example, the ROI pooling layer (a type of downsampling layer) can be used to obtain a feature map of the same size as the Conv-5 layer. The feature map obtained by Conv-8 is smaller and has high semantic information. It can be upsampled to increase its size, such as using nearest neighbor interpolation, to obtain a feature map of the same size as Conv-5.

[0086] In order to accurately train the initial feature processing model, in the bill image classification method provided in Example 1 of the present application, a historical bill image set is obtained; each historical bill image in the historical bill image set is expanded to obtain a training set, wherein the expansion processing includes at least one of the following: rotation, scaling, and flipping; each training sample in the training set is associated with a labeled category; each training sample is a historical bill image or an expanded historical bill image; in combination with a preset classifier, the initial feature processing model is iterated using the training set to obtain a preset feature processing model, wherein the process of iterating the initial feature processing model using the training set includes: inputting each training sample into the initial feature processing model in turn to obtain a prediction weight feature map; inputting each prediction weight feature map into the preset classifier in turn to obtain a prediction category; based on each prediction category and the labeled category corresponding to each prediction category, the model parameters in the initial feature processing model are adjusted until the loss value determined based on the prediction category and the labeled category is less than the preset loss threshold.

[0087] In an embodiment of the present invention, a large number of historical bill images can be collected to form a diverse image collection. These images may include, but are not limited to, checks, bills, and receipts, encompassing bill instances with varying fonts, patterns, sizes, and backgrounds. Furthermore, to enhance the diversity of the training dataset, each image in the historical bill image collection can be subjected to extended processing, including but not limited to rotation, scaling, and flipping. For example, bill images can be rotated at random angles to simulate the diverse placement of bills in practice; bill images can be resized to train the model for inputs of varying sizes; and bill images can be flipped horizontally or vertically to increase the richness of the dataset. This not only increases the number of training samples but also improves their diversity, allowing the model to be exposed to more complex and varied bill image conditions during training.

[0088] In this embodiment of the present invention, each bill image generated during the expansion process is added to the training set, forming a comprehensive training dataset along with the unexpanded historical bill images. Each training sample (whether a historical bill image or an expanded image) is associated with a labeled category, which is manually assigned or provided by a pre-trained model based on the image's actual type (e.g., check, bill, receipt, etc.).

[0089] In an embodiment of the present invention, the initial feature processing model is iteratively trained using a training set in combination with a preset classifier to obtain a final "preset feature processing model".

[0090] Specifically, each training sample in the training set is input into the initial feature processing model in turn. The model extracts and weights the features of the bill image through the multi-scale attention feature fusion structure to obtain a predicted weight feature map. Then, each predicted weight feature map is input into the preset classifier, and the predicted category is obtained through the fully connected layer and softmax layer of the classifier. After that, the loss value (such as cross entropy loss) is calculated based on each predicted category and the corresponding labeled category. If the loss value exceeds the preset loss threshold, the model parameters in the initial feature processing model will be adjusted through the back propagation algorithm to reduce the prediction error. This process will be iterated repeatedly until the loss value is lower than the preset threshold and the model achieves a better classification effect.

[0091] In this embodiment, by utilizing and expanding a collection of historical bill images, the amount of training data is increased and the diversity of the dataset is enhanced, enabling the model to learn a wider range of bill image features during training. Combined with iterative training using a pre-set classifier, the initial feature processing model gradually optimizes its parameters, learning more accurate bill classification capabilities. The resulting pre-set feature processing model not only efficiently processes various bill image types, but also exhibits high classification accuracy and robustness, enabling it to cope with the diverse variations and challenges of bill images in real-world applications.

[0092] Optionally, the multi-scale attention feature fusion structure includes at least: a first convolutional layer, a second convolutional layer, and a third convolutional layer. In order to accurately obtain the target weight feature map, in the bill image classification method provided in Example 1 of the present application, the first convolutional layer is used to extract features of the target bill image to obtain a first feature map; the second convolutional layer is used to extract features of the first feature map to obtain a second feature map, wherein the scale of the second feature map is smaller than the scale of the first feature map; the third convolutional layer is used to extract features of the second feature map to obtain a third feature map, wherein the scale of the third feature map is smaller than the scale of the second feature map; the spatial attention structure is used to process the first feature map to obtain a spatial weight feature map; the spatial attention structure and the channel attention structure are used to process the second feature map to obtain a fused weight feature map; the channel attention structure is used to process the third feature map to obtain a channel weight feature map; the fusion structure in the multi-scale attention feature fusion structure is used to fuse the spatial weight feature map, the fused weight feature map, and the channel weight feature map to obtain the target weight feature map.

[0093] In this embodiment of the present invention, the first convolutional layer (e.g., Conv-2) can be used for feature extraction. For example, the preprocessed target bill image is input into the first convolutional layer of the deep learning model for preliminary feature extraction. This layer typically contains a smaller convolution kernel that can capture low-level features such as edges and textures of the bill image. The output result is the first feature map.

[0094] Then, a second convolutional layer (such as Conv-5) is used for feature extraction. For example, the first feature map is passed to the second convolutional layer for deeper feature extraction. The kernel size and stride of the second convolutional layer are usually increased to capture more complex patterns and information, such as specific fonts or patterns. Due to the pooling operation of this layer, the scale of the second feature map is smaller than the first feature map, but its semantic information is richer.

[0095] After that, a third convolutional layer (e.g., Conv-8) is used for feature extraction. For example, the third convolutional layer further deepens the feature extraction hierarchy, capturing more abstract features such as the layout structure or key information areas of the bill. Compared to the second feature map, the third feature map has a smaller scale but a higher semantic depth.

[0096] In an embodiment of the present invention, the first feature map can be weighted using a spatial attention structure. This structure can learn the spatial importance of different regions in the bill image, helping the model focus on key visual elements, such as the seal or signature area, and ignoring unimportant background information. In addition, the second feature map can be processed using a spatial attention structure and a channel attention structure. After the second feature map is weighted through SAM and CAM, a fused weight feature map is obtained. SAM helps the model pay attention to the position information in the second feature map, while CAM further enhances the representation ability of different feature channels, enabling the model to more accurately understand the detailed features of the bill, such as the font changes of the amount digits. At the same time, the third feature map can be processed using a channel attention structure to obtain a channel weight feature map. At this stage, the physical size information of the bill image has been greatly reduced, while the density of semantic information (such as bill category, amount, etc.) has been significantly improved. CAM can highlight these high-level semantic features, enabling the model to more accurately identify and classify.

[0097] In this embodiment of the present invention, the fusion structure in the multi-scale attention feature fusion structure is used to fuse the three weighted feature maps (spatial weight feature map, fusion weight feature map, and channel weight feature map) mentioned above. This ensures that the model can simultaneously utilize low-level texture information, mid-level pattern and position information, and high-level semantic information to obtain a comprehensive and weighted "target weight feature map." The target weight feature map will be provided as input to the classifier for the final bill category prediction.

[0098] In this embodiment, by using a multi-scale convolutional layer for feature extraction and combining SAM and CAM for feature weighting, multi-level and multi-scale feature information can be extracted from the target bill image, while highlighting key areas and semantic features. This hierarchical and multi-scale feature extraction and weighting enables the model to understand the physical properties and semantic content of the bill at the same time, greatly improving the accuracy of classification. Finally, these weighted feature maps are integrated into a target weighted feature map through a fusion structure, which not only retains the comprehensive information of the bill image, but also strengthens the key features through weight adjustment. The classification performance of the final model is significantly improved, and it can cope with complex and changeable bill images, improving the efficiency and reliability of financial institutions' automated bill processing.

[0099] Figure 4 Schematic diagram of a multi-scale feature fusion framework based on attention according to embodiment 1 of the present invention, Figure 4As shown, the first convolutional layer Conv-2 can be used to extract features from the input bill image (Img), and then the second convolutional layer Conv-5 can be used to convolve the feature map transmitted by the first convolutional layer Conv-2, and then the third convolutional layer Conv-8 can be used to convolve the feature map transmitted by the second convolutional layer Conv-5 to obtain feature maps of different scales. The feature map output by the first convolutional layer Conv-2 can be processed by SAM to obtain a spatial weight feature map Map. The feature map output by the second convolutional layer Conv-5 can be processed by SAM and CAM to obtain a fused weight feature map Map. The feature map output by the third convolutional layer Conv-8 can be processed by CAM to obtain a channel weight feature map Map. Then, the spatial weight feature map, the fused weight feature map, and the channel weight feature map are fused (Fused), and the fused feature map is processed by the softmax layer.

[0100] exist Figure 4 In [1], multi-scale feature extraction can generate feature maps of varying sizes, enabling better integration with the attention mechanism. Low-level feature maps are larger and contain richer spatial information, making them suitable for learning spatial information from different regions using the spatial attention mechanism. High-level feature maps become smaller with convolutional pooling, gradually reducing spatial information while enriching the semantic information represented by different channels. Therefore, using channel attention to learn weights for different channels allows the network model to prioritize effective semantic features in the bill classification process, improving the network's performance. The fused feature maps are then fed into the classifier for prediction, yielding the model's final classification result.

[0101] In order to improve the accuracy of determining the spatial weight feature map, in the bill image classification method provided in Example 1 of the present application, the first feature map is subjected to maximum pooling operation, convolution operation and average pooling operation respectively to obtain a first sub-feature map, a second sub-feature map and a third sub-feature map; the first sub-feature map, the second sub-feature map and the third sub-feature map are spliced according to the channel dimension to obtain a multi-channel feature map; the multi-channel feature map is convolved to obtain a first single-channel feature map, and the first single-channel feature map is processed with an activation function to obtain a first spatial attention map; the first spatial attention map is dot-multiplied with the first feature map to obtain a spatial weight feature map.

[0102] In this embodiment of the present invention, the model automatically calculates the importance of each feature through a self-attention mechanism, assigning different weights based on importance to highlight key areas. This embodiment can use spatial attention modules and channel attention modules for weight assignment. Spatial attention learns weights based on the spatial information of the image, reducing background weights and allowing the network to focus more on key elements such as the text, seal, and signature information in the bill image. Channel attention, on the other hand, learns the semantic information of different channels in the feature map.

[0103] In an embodiment of the present invention, a maximum pooling (MaxPooling) operation can be first performed on the first feature map extracted by the lower convolutional layer (such as Conv-2) to extract the maximum value feature from each local area, which helps the model capture significant visual elements in the bill image, such as seals, etc., and the output result is the first sub-feature map. In addition, the first feature map is further extracted through the convolution (Conv) operation, and a set of learned convolution kernels can be used to generate a new feature map to capture more complex local feature patterns, such as specific fonts or patterns. The output of the convolution operation is the second sub-feature map. At the same time, the first feature map is subjected to an average pooling (AvgPooling) operation, which generates a feature map by calculating the average value of the local area, which helps the model learn evenly distributed features and reduce the impact of noise. The output result is the third sub-feature map.

[0104] Then, the first, second, and third sub-feature maps are concatenated based on the channel dimension to generate a multi-channel feature map. Each sub-feature map represents a different feature extraction method, and concatenating them together can provide more comprehensive bill image information.

[0105] Then, the constructed multi-channel feature map can be convolved to integrate the multi-channel information into a single-channel feature map. The single-channel feature map can better summarize different types of local features, making it easier to process in the subsequent spatial attention mechanism. The first single-channel feature map can be processed using an activation function (such as Sigmoid) to generate the first spatial attention map. The use of the activation function can map the values in the feature map to a nonlinear space, thereby highlighting the features of the key areas in the bill image and reducing the influence of the background.

[0106] Next, the first spatial attention map is multiplied with the first feature map to generate a spatial weighted feature map. Through this dot product, the weights in the spatial attention map are applied to the corresponding positions in the first feature map. This essentially adjusts the feature weights at each position in the feature map, allowing the model to focus more on key information areas of the bill, such as the amount, signature, and date.

[0107] In this embodiment, key visual elements in bill images, such as seals, signatures, and numbers, can be identified and enhanced from the first feature map. The generation and application of spatial attention maps essentially enables the model to focus more on image regions with classification value through a weight adjustment mechanism, thereby demonstrating higher accuracy and robustness in bill image classification tasks.

[0108] Figure 5 Schematic diagram of the spatial attention module processing flow according to embodiment 1 of the present invention. Figure 5 As shown, we can first Convolution (Conv) and pooling operations (MaxPooling and AvgPooling) are performed to further extract spatial information, where: Represents the feature map output by Conv-2. The convolution operation Conv is a 1×1 convolution with 1 convolution kernel and the same feature map. Perform maximum pooling MaxPooling and average pooling AvgPooling respectively to extract the spatial information of different channels, and then obtain the three feature maps Perform channel dimension concatenation to obtain feature maps After that, the feature fusion is performed through the convolution operation Conv to obtain the feature map And through the activation function (Sigmoid) the feature map Processing is performed to obtain a spatial attention map Finally, the spatial attention map and the original feature map Perform spatial point multiplication to obtain the feature map of the fused spatial weights.

[0109] In order to improve the accuracy of determining the fused weight feature map, in the bill image classification method provided in Example 1 of the present application, the second feature map is copied to obtain a second feature copy map; the second feature map is processed using a spatial attention structure to obtain a second spatial attention map, and the second spatial attention map is dot-multiplied with the second feature map to obtain a first weight feature map; the second feature copy map is processed using a channel attention structure to obtain a first channel attention map, and the first channel attention map is dot-multiplied with the second feature copy map to obtain a second weight feature map; the first weight feature map and the second weight feature map are fused to obtain a fused weight feature map.

[0110] In an embodiment of the present invention, the second feature map output by the second convolutional layer can be copied to obtain a second feature copy map. In this way, the model can independently apply spatial and channel attention mechanisms to the original image and the copy map, thereby extracting features from two perspectives.

[0111] In an embodiment of the present invention, a spatial attention algorithm (SAM) is used to process the second feature map to generate a second spatial attention map. The purpose of SAM is to learn the spatial weights of different locations in the bill image, helping the model focus on high-information-density areas, such as signatures and amounts, while reducing the weights of background or other irrelevant areas. Subsequently, the second spatial attention map is dot-multiplied with the second feature map to generate a first weighted feature map in a weighted manner, thereby enhancing the features of the key areas.

[0112] In this embodiment of the present invention, the second feature copy map is processed using a channel attention architecture (CAM) to generate a first channel attention map. The purpose of CAM is to learn the weights between different feature channels, emphasizing channels that carry more semantic information, such as font style and pattern complexity. Next, the first channel attention map is dot-multiplied with the second feature copy map to generate a second weighted feature map. In this process, the model further enhances its understanding of the high-level features of the bill image.

[0113] In an embodiment of the present invention, the first weighted feature map and the second weighted feature map are fused. Fusion can be achieved in a variety of ways, such as simple summation of feature maps, concatenation followed by convolution, etc. In this way, weighted information in both spatial and channel dimensions can be integrated to obtain a more comprehensive fused weighted feature map.

[0114] In this embodiment, the second feature map and its copy are processed by a dual attention mechanism (SAM and CAM), and finally a fusion weight feature map is generated, which can significantly improve the performance of bill image classification. Specifically, the second spatial attention map adjusts the weight of the spatial dimension so that the model can more accurately locate the key elements on the bill; while the first channel attention map adjusts the weight of the channel dimension so that the model can more deeply understand the intrinsic semantic information of the bill. The fusion of these two weighted feature maps not only retains the low-level texture and shape features, but also highlights the high-level semantic and pattern features, greatly improving the classification accuracy and robustness of the model. In practical applications, this method can effectively adapt to the changes of various bill images, reduce feature loss and noise interference, and thus play an important role in the automated bill processing system, improving the operational efficiency and customer experience of financial institutions.

[0115] In order to improve the accuracy of determining the channel weight feature map, in the bill image classification method provided in Example 1 of the present application, the third feature map is subjected to maximum pooling operations and average pooling operations respectively to obtain a fourth sub-feature map and a fifth sub-feature map; the fourth sub-feature map is convolved to obtain a first single-channel map, and the fifth sub-feature map is convolved to obtain a second single-channel map; the first single-channel map and the second single-channel map are spliced to obtain a spliced single-channel map; the spliced single-channel map is convolved to obtain a second single-channel feature map, and the second single-channel feature map is processed with an activation function to obtain a second channel attention map; the second channel attention map is dot-multiplied with the third feature map to obtain a channel weight feature map.

[0116] In an embodiment of the present invention, the third feature map output by the third convolutional layer can be subjected to a maximum pooling (MaxPooling) operation and an average pooling (AvgPooling) operation to obtain a fourth sub-feature map and a fifth sub-feature map, respectively. The purpose of these two pooling operations is to extract local statistical information of different channels in the feature map. Among them, maximum pooling focuses on the maximum value of the local area and can capture significant features; average pooling calculates the average value of the local area, which helps the model learn more evenly distributed feature patterns.

[0117] Then, convolution operations are performed on the fourth and fifth sub-feature maps, such as using a 1×1 convolution kernel (commonly known as point convolution), to reduce the number of channels to 1, generating the first and second single-channel maps. Generating single-channel maps helps the model extract key information from the results of different pooling operations, while reducing the amount of computation and parameters, making the subsequent channel attention mechanism processing more efficient.

[0118] Afterwards, the first single-channel image and the second single-channel image are concatenated to obtain a concatenated single-channel image. The concatenation operation can integrate the feature information extracted by different pooling methods to form a richer feature representation, providing diverse input for the channel attention mechanism.

[0119] Next, a convolution operation is performed on the concatenated single-channel image to produce a second single-channel feature map. This process further refines the information in the feature map, paving the way for subsequent channel weight assignment. The second single-channel feature map is then processed using an activation function (such as Sigmoid) to produce a second channel attention map. The use of an activation function converts the numerical values in the feature map into weighted values, highlighting features of the bill image that are valuable for classification while suppressing irrelevant background or noise.

[0120] Finally, a dot product operation is performed on the second channel attention map and the third feature map to obtain a channel weight feature map. This operation essentially assigns the weights in the channel attention map to each channel of the third feature map, strengthening the representation of important features and allowing the model to focus more on the key information of the bill during the classification process, such as the semantic features of numbers, text, and seals.

[0121] In this embodiment, different local statistical features are extracted from the third feature map by using maximum pooling and average pooling operations, and a single-channel representation is generated through point convolution. The channel attention mechanism is further applied to finally generate a channel weight feature map. This method achieves weighted adjustment of features in the channel dimension, which not only enhances the model's ability to understand the semantic features of bill images, but also improves the accuracy and robustness of classification. In particular, the generation of the channel weight feature map enables the model to focus more on features that are critical to bill classification, such as text, numbers, and seals, thereby demonstrating more outstanding performance in processing complex bill images.

[0122] Figure 6 Schematic diagram of the channel attention module processing flow according to embodiment 1 of the present invention. Figure 6 As shown, we can first Perform pooling operations (MaxPooling and AvgPooling), where Represents the feature map output by Conv-8. Then, the two feature maps obtained Perform convolution operations (Conv) separately to obtain feature maps For two feature maps Perform concatenation and Concat to obtain the feature map After that, the feature fusion is performed through the convolution operation Conv to obtain the feature map And through the activation function (Sigmoid) the feature map Process and get the channel attention map Finally, the channel attention map and the original feature map Perform spatial point multiplication to obtain the feature map of the fused channel weights.

[0123] In this embodiment, as the number of convolution and pooling layers increases, the number of feature map channels gradually increases, spatial information gradually decreases, and channel semantic information gradually increases. The channel attention map learns the semantic information represented by different channels, enriching the semantic information of high-level feature maps. Using the attention mechanism makes the network model more effective.

[0124] In order to improve the accuracy of determining the target weight feature map, in the bill image classification method provided in Example 1 of the present application, the spatial weight feature map is downsampled to obtain a downsampled spatial weight feature map with the same scale as the fusion weight feature map; the channel weight feature map is upsampled to obtain an upsampled channel weight feature map with the same scale as the fusion weight feature map; and the downsampled spatial weight feature map, the fusion weight feature map and the upsampled channel weight feature map are fused using a fusion structure to obtain the target weight feature map.

[0125] In an embodiment of the present invention, since the spatial weight feature map is rich in spatial information but has a larger scale than the fusion weight feature map, a downsampling operation (such as ROI pooling or maximum pooling, average pooling, etc.) can be used to scale the spatial weight feature map to obtain a downsampled spatial weight feature map of the same scale as the fusion weight feature map. The purpose of the downsampling operation is to reduce the spatial resolution of the feature map so that it can be consistent in scale with the fusion weight feature map, which is convenient for subsequent fusion processing. In addition, since the channel weight feature map has undergone deep convolution processing, the scale is small but the semantic information is dense, an upsampling operation (such as nearest neighbor interpolation, bilinear interpolation, etc.) can be used to scale the channel weight feature map to obtain an upsampled channel weight feature map of the same scale as the fusion weight feature map. The upsampling operation can increase the resolution of the feature map, so that the channel weight feature map can maintain the corresponding details when fused with the spatial weight feature map, and ensure that the fused feature map can simultaneously reflect the weighted information of space and channel. The downsampled spatial weight feature map, the fused weight feature map, and the upsampled channel weight feature map are then fused using a fusion structure (such as concat or element-by-element addition) to generate a target weight feature map. The fusion structure aims to integrate feature maps processed at different scales and using different attention mechanisms to form a high-quality feature representation that combines spatial information, fused weight information, and channel semantic information. This process enables the model to understand the characteristics of bill images from multiple perspectives, improving classification accuracy.

[0126] In this embodiment, through scale adjustment and feature fusion, the spatial weight feature map, fusion weight feature map, and channel weight feature map are effectively combined at the same scale to generate a target weight feature map. This process solves the problem of feature maps of different scales being unable to be directly fused, enabling the model to simultaneously consider the key spatial regions and semantic features of bill images, as well as the comprehensive information extracted through the fusion mechanism, when classifying bill images. Ultimately, the generation of the target weight feature map significantly enhances the model's classification performance, improving the accuracy and robustness of bill image classification.

[0127] The bill image classification method provided in the embodiments of the present application, through multi-scale feature extraction and an adaptive attention mechanism, can more accurately capture key information in bill images and reduce the misclassification rate. Furthermore, the multi-scale attention mechanism uses different attention mechanism learning weights for feature maps at different scales, effectively addressing issues such as the diversity of bill image formats and noise interference, allowing the network model to focus more closely on important areas and improving the stability and robustness of classification. Furthermore, this method can be widely applied to the classification of different types of bill images, such as checks, bills of exchange, and receipts, and exhibits strong generalization capabilities.

[0128] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0129] Example 2

[0130] The present application also provides a bill image classification device. It should be noted that the bill image classification device of the present application can be used to execute the bill image classification method provided in the present application. The bill image classification device provided in the present application is described below.

[0131] According to an embodiment of the present application, a device for implementing the above-mentioned bill image classification method is also provided. Figure 7 is a schematic diagram of an optional bill image classification device according to an embodiment of the present invention, such as Figure 7 As shown, the classification device may include: a first processing unit 70 , a second processing unit 71 , and a classification unit 72 .

[0132] The first processing unit 70 is used to pre-process the bill image to obtain a target bill image;

[0133] A second processing unit 71 is configured to process the target bill image using a preset feature processing model to obtain a target weighted feature map, wherein the preset feature processing model includes at least a multi-scale attention feature fusion structure; the multi-scale attention feature fusion structure is configured to extract feature maps of the target bill image at different scales and assign weights to the feature maps at different scales to obtain a target weighted feature map;

[0134] The classification unit 72 is used to classify the target weight feature map using a preset classifier to obtain the image category of the bill image.

[0135] The bill image classification device provided in the embodiment of the present application can extract feature maps of the bill image at different scales by adding a multi-scale attention feature fusion structure to the preset feature processing model, and assign weights to the feature maps at different scales to obtain a target weight feature map of the fusion weight. Thereafter, the target weight feature map is classified by a preset classifier to accurately obtain the image category of the bill image, thereby achieving the technical effect of improving the accuracy and robustness of the bill image classification, and further solving the technical problem of low accuracy in classifying bill images in related technologies.

[0136] Optionally, the first processing unit includes: performing grayscale processing on the bill image to obtain a grayscale image; and normalizing the grayscale image to obtain a target bill image.

[0137] Optionally, the classification device also includes: a first construction unit, used to construct a multi-scale attention feature fusion structure before using a preset feature processing model to process the target bill image and obtain a target weight feature map, wherein the multi-scale attention feature fusion structure includes at least: multiple layers of convolution layers, an attention structure connected to each convolution layer, and a fusion structure; a second construction unit, used to construct an initial feature processing model based on the multi-scale attention feature fusion structure; a first training unit, used to train the initial feature processing model in combination with a preset classifier to obtain a preset feature processing model.

[0138] Optionally, the first training unit includes: a first acquisition module for acquiring a historical bill image set; a first expansion module for performing expansion processing on each historical bill image in the historical bill image set to obtain a training set, wherein the expansion processing includes at least one of the following: rotation, scaling, and flipping; each training sample in the training set is associated with a labeled category; each training sample is a historical bill image or an expanded historical bill image; a first iteration module for combining a preset classifier and using the training set to iterate the initial feature processing model to obtain a preset feature processing model, wherein the process of iterating the initial feature processing model using the training set includes: inputting each training sample into the initial feature processing model in turn to obtain a prediction weight feature map; inputting each prediction weight feature map into the preset classifier in turn to obtain a prediction category; based on each prediction category and the labeled category corresponding to each prediction category, adjusting the model parameters in the initial feature processing model until the loss value determined based on the prediction category and the labeled category is less than the preset loss threshold.

[0139] Optionally, the multi-scale attention feature fusion structure includes at least: a first convolutional layer, a second convolutional layer, and a third convolutional layer, and the second processing unit includes: a first extraction module, used to use the first convolutional layer to perform feature extraction on the target bill image to obtain a first feature map; a second extraction module, used to use the second convolutional layer to perform feature extraction on the first feature map to obtain a second feature map, wherein the scale of the second feature map is smaller than the scale of the first feature map; a third extraction module, used to use the third convolutional layer to perform feature extraction on the second feature map to obtain a third feature map, wherein the scale of the third feature map is smaller than the scale of the second feature map; a first processing module, used to use the spatial attention structure to process the first feature map to obtain a spatial weight feature map; a second processing module, used to use the spatial attention structure and the channel attention structure to process the second feature map to obtain a fused weight feature map; a third processing module, used to use the channel attention structure to process the third feature map to obtain a channel weight feature map; a first fusion module, used to use the fusion structure in the multi-scale attention feature fusion structure to fuse the spatial weight feature map, the fused weight feature map and the channel weight feature map to obtain a target weight feature map.

[0140] Optionally, the first processing module includes: a first operation submodule, used to perform maximum pooling operation, convolution operation and average pooling operation on the first feature map, respectively, to obtain a first sub-feature map, a second sub-feature map and a third sub-feature map; a first splicing submodule, used to splice the first sub-feature map, the second sub-feature map and the third sub-feature map according to the channel dimension to obtain a multi-channel feature map; a first convolution submodule, used to convolve the multi-channel feature map to obtain a first single-channel feature map, and process the first single-channel feature map using an activation function to obtain a first spatial attention map; a first point multiplication submodule, used to perform point multiplication on the first spatial attention map and the first feature map to obtain a spatial weight feature map.

[0141] Optionally, the second processing module includes: a first copying submodule, used to copy the second feature map to obtain a second feature copy map; a first processing submodule, used to process the second feature map using a spatial attention structure to obtain a second spatial attention map, and perform dot multiplication of the second spatial attention map with the second feature map to obtain a first weight feature map; a second processing submodule, used to process the second feature copy map using a channel attention structure to obtain a first channel attention map, and perform dot multiplication of the first channel attention map with the second feature copy map to obtain a second weight feature map; a third processing submodule, used to perform fusion processing on the first weight feature map and the second weight feature map to obtain a fused weight feature map.

[0142] Optionally, the third processing module includes: a second operation submodule, used to perform maximum pooling operations and average pooling operations on the third feature map, respectively, to obtain a fourth sub-feature map and a fifth sub-feature map; a second convolution submodule, used to convolve the fourth sub-feature map to obtain a first single-channel map, and to convolve the fifth sub-feature map to obtain a second single-channel map; a second splicing submodule, used to splice the first single-channel map and the second single-channel map to obtain a spliced single-channel map; a third convolution submodule, used to convolve the spliced single-channel map to obtain a second single-channel feature map, and process the second single-channel feature map using an activation function to obtain a second channel attention map; a second point multiplication submodule, used to perform point multiplication on the second channel attention map and the third feature map to obtain a channel weight feature map.

[0143] Optionally, the first fusion module includes: a first downsampling submodule, used to downsample the spatial weight feature map to obtain a downsampled spatial weight feature map with the same scale as the fusion weight feature map; a first upsampling submodule, used to upsample the channel weight feature map to obtain an upsampled channel weight feature map with the same scale as the fusion weight feature map; a first fusion submodule, used to fuse the downsampled spatial weight feature map, the fusion weight feature map and the upsampled channel weight feature map using a fusion structure to obtain a target weight feature map.

[0144] The above-mentioned classification device may also include a processor and a memory. The above-mentioned first processing unit 70, second processing unit 71, classification unit 72, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0145] The processor includes a kernel that retrieves the corresponding program unit from the memory. One or more kernels can be provided. By adjusting kernel parameters, a preset classifier is used to classify the target weight feature map to obtain the image category of the bill image.

[0146] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0147] It should be noted that the first processing unit 70, the second processing unit 71, and the classification unit 72 correspond to steps S201 to S203 in Example 1. The examples and application scenarios implemented by the above units and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above units can also be part of the device and can be run in the computer terminal 10 provided in Example 1.

[0148] Example 3

[0149] An embodiment of the present application may provide an electronic device, Figure 8 1 is a structural block diagram of an electronic device according to an embodiment of the present invention. Figure 8 As shown, the electronic device may include: one or more ( Figure 8 Only one is shown) processor 802, memory 804, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0150] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the bill image classification method and device in the embodiments of the present application. The processor executes the software programs and modules stored in the memory to execute various functional applications and data processing, thereby implementing the above-mentioned bill image classification method. The memory may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include memory remotely located relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0151] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: pre-process the bill image to obtain a target bill image; use a preset feature processing model to process the target bill image to obtain a target weight feature map, wherein the preset feature processing model at least includes: a multi-scale attention feature fusion structure; the multi-scale attention feature fusion structure is used to extract feature maps of the target bill image at different scales, and weight the feature maps at different scales to obtain a target weight feature map; use a preset classifier to classify the target weight feature map to obtain the image category of the bill image.

[0152] The processor can also call the information and application program stored in the memory through the transmission device to perform the following steps: grayscale processing of the bill image to obtain a grayscale image; normalizing the grayscale image to obtain a target bill image.

[0153] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: construct a multi-scale attention feature fusion structure, wherein the multi-scale attention feature fusion structure includes at least: multiple convolution layers, an attention structure connected to each convolution layer, and a fusion structure; based on the multi-scale attention feature fusion structure, construct an initial feature processing model; combine the preset classifier to train the initial feature processing model to obtain a preset feature processing model.

[0154] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtaining a historical bill image set; performing expansion processing on each historical bill image in the historical bill image set to obtain a training set, wherein the expansion processing includes at least one of the following: rotation, scaling, and flipping; each training sample in the training set is associated with a labeled category; each training sample is a historical bill image or an expanded historical bill image; in combination with a preset classifier, the initial feature processing model is iterated using the training set to obtain a preset feature processing model, wherein the process of iterating the initial feature processing model using the training set includes: inputting each training sample into the initial feature processing model in turn to obtain a prediction weight feature map; inputting each prediction weight feature map into the preset classifier in turn to obtain a prediction category; based on each prediction category and the labeled category corresponding to each prediction category, the model parameters in the initial feature processing model are adjusted until the loss value determined based on the prediction category and the labeled category is less than the preset loss threshold.

[0155] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: using the first convolution layer to extract features of the target bill image to obtain a first feature map; using the second convolution layer to extract features of the first feature map to obtain a second feature map, wherein the scale of the second feature map is smaller than the scale of the first feature map; using the third convolution layer to extract features of the second feature map to obtain a third feature map, wherein the scale of the third feature map is smaller than the scale of the second feature map; using the spatial attention structure to process the first feature map to obtain a spatial weight feature map; using the spatial attention structure and the channel attention structure to process the second feature map to obtain a fused weight feature map; using the channel attention structure to process the third feature map to obtain a channel weight feature map; using the fusion structure in the multi-scale attention feature fusion structure to fuse the spatial weight feature map, the fused weight feature map and the channel weight feature map to obtain a target weight feature map.

[0156] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: perform maximum pooling operations, convolution operations, and average pooling operations on the first feature map to obtain a first sub-feature map, a second sub-feature map, and a third sub-feature map; splice the first sub-feature map, the second sub-feature map, and the third sub-feature map according to the channel dimension to obtain a multi-channel feature map; convolve the multi-channel feature map to obtain a first single-channel feature map, and use an activation function to process the first single-channel feature map to obtain a first spatial attention map; perform dot multiplication on the first spatial attention map and the first feature map to obtain a spatial weight feature map.

[0157] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: copy the second feature map to obtain a second feature copy map; use the spatial attention structure to process the second feature map to obtain a second spatial attention map, and perform dot multiplication of the second spatial attention map and the second feature map to obtain a first weight feature map; use the channel attention structure to process the second feature copy map to obtain a first channel attention map, and perform dot multiplication of the first channel attention map and the second feature copy map to obtain a second weight feature map; fuse the first weight feature map and the second weight feature map to obtain a fused weight feature map.

[0158] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: performing maximum pooling operations and average pooling operations on the third feature map to obtain a fourth sub-feature map and a fifth sub-feature map; convolving the fourth sub-feature map to obtain a first single-channel map, and convolving the fifth sub-feature map to obtain a second single-channel map; splicing the first single-channel map and the second single-channel map to obtain a spliced single-channel map; convolving the spliced single-channel map to obtain a second single-channel feature map, and processing the second single-channel feature map with an activation function to obtain a second channel attention map; performing a dot multiplication on the second channel attention map and the third feature map to obtain a channel weight feature map.

[0159] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: downsampling the spatial weight feature map to obtain a downsampled spatial weight feature map with the same scale as the fusion weight feature map; upsampling the channel weight feature map to obtain an upsampled channel weight feature map with the same scale as the fusion weight feature map; using a fusion structure to fuse the downsampled spatial weight feature map, the fusion weight feature map and the upsampled channel weight feature map to obtain a target weight feature map.

[0160] An embodiment of the present application provides a solution for classifying bill images. By adding a multi-scale attention feature fusion structure to a preset feature processing model, feature maps of the bill image at different scales can be extracted and weighted to obtain a target weight feature map of the fusion weights. This target weight feature map is then classified by a preset classifier to accurately determine the image category of the bill image, thereby achieving the technical effect of improving the accuracy and robustness of bill image classification and solving the technical problem of low accuracy in bill image classification in related technologies.

[0161] It can be understood by those skilled in the art that Figure 8 The structure shown is for illustration only, and the electronic device may also be a terminal device such as a smart phone, a tablet computer, a PDA, or a mobile Internet device (MID). Figure 8 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 8 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 8 Different configurations shown.

[0162] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0163] Example 4

[0164] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the bill image classification method provided in the first embodiment.

[0165] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0166] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the method for classifying bill images.

[0167] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0168] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0170] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0171] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0172] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0173] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A bill image classification method, characterized in that: include: Preprocess the bill image to obtain the target bill image; The target bill image is processed using a preset feature processing model to obtain a target weight feature map, wherein the preset feature processing model at least includes: a multi-scale attention feature fusion structure; the multi-scale attention feature fusion structure is used to extract feature maps of the target bill image at different scales, and weight the feature maps at different scales to obtain the target weight feature map; The target weight feature map is classified using a preset classifier to obtain the image category of the bill image.

2. The classification method according to claim 1, characterized in that The steps of preprocessing the bill image to obtain the target bill image include: Performing grayscale processing on the bill image to obtain a grayscale image; The grayscale image is normalized to obtain the target bill image.

3. The classification method according to claim 1, characterized in that Before using the preset feature processing model to process the target bill image to obtain the target weight feature map, the method further includes: Constructing the multi-scale attention feature fusion structure, wherein the multi-scale attention feature fusion structure at least includes: multiple convolutional layers, an attention structure connected to each convolutional layer, and a fusion structure; Based on the multi-scale attention feature fusion structure, an initial feature processing model is constructed; The initial feature processing model is trained in combination with the preset classifier to obtain the preset feature processing model.

4. The classification method according to claim 3, characterized in that The step of training the initial feature processing model in combination with the preset classifier to obtain the preset feature processing model includes: Get the historical bill image collection; Performing expansion processing on each historical bill image in the historical bill image set to obtain a training set, wherein the expansion processing includes at least one of the following: rotation, scaling, and flipping; each training sample in the training set is associated with a labeled category; and each training sample is the historical bill image or an expanded historical bill image; In combination with the preset classifier, the initial feature processing model is iterated using the training set to obtain the preset feature processing model, wherein the process of iterating the initial feature processing model using the training set includes: Inputting each of the training samples into the initial feature processing model in sequence to obtain a prediction weight feature map; Inputting each of the prediction weight feature maps into the preset classifier in sequence to obtain a prediction category; Based on each of the predicted categories and the labeled categories corresponding to each of the predicted categories, model parameters in the initial feature processing model are adjusted until a loss value determined based on the predicted category and the labeled category is less than a preset loss threshold.

5. The classification method according to claim 1, characterized in that The multi-scale attention feature fusion structure includes at least: a first convolutional layer, a second convolutional layer, and a third convolutional layer. The step of processing the target bill image using a preset feature processing model to obtain a target weight feature map includes: Using the first convolutional layer to extract features from the target bill image to obtain a first feature map; Performing feature extraction on the first feature map using the second convolutional layer to obtain a second feature map, wherein the scale of the second feature map is smaller than the scale of the first feature map; Performing feature extraction on the second feature map using the third convolutional layer to obtain a third feature map, wherein a scale of the third feature map is smaller than a scale of the second feature map; Processing the first feature map using a spatial attention structure to obtain a spatial weight feature map; Processing the second feature map using the spatial attention structure and the channel attention structure to obtain a fused weight feature map; Processing the third feature map using the channel attention structure to obtain a channel weight feature map; The spatial weight feature map, the fusion weight feature map and the channel weight feature map are fused using the fusion structure in the multi-scale attention feature fusion structure to obtain the target weight feature map.

6. The classification method according to claim 5, characterized in that: The step of processing the first feature map using a spatial attention structure to obtain a spatial weight feature map includes: Performing a maximum pooling operation, a convolution operation, and an average pooling operation on the first feature map to obtain a first sub-feature map, a second sub-feature map, and a third sub-feature map; splicing the first sub-feature map, the second sub-feature map, and the third sub-feature map according to the channel dimension to obtain a multi-channel feature map; Convolving the multi-channel feature map to obtain a first single-channel feature map, and processing the first single-channel feature map with an activation function to obtain a first spatial attention map; Perform a dot product of the first spatial attention map and the first feature map to obtain the spatial weight feature map.

7. The classification method according to claim 5, characterized in that The step of processing the second feature map using the spatial attention structure and the channel attention structure to obtain a fused weight feature map includes: Copying the second characteristic map to obtain a second characteristic copy map; Processing the second feature map using the spatial attention structure to obtain a second spatial attention map, and performing a dot product between the second spatial attention map and the second feature map to obtain a first weighted feature map; The second feature copy map is processed using the channel attention structure to obtain a first channel attention map, and the first channel attention map is dot-multiplied with the second feature copy map to obtain a second weighted feature map; The first weight feature map and the second weight feature map are fused to obtain the fused weight feature map.

8. The classification method according to claim 5, characterized in that: The step of processing the third feature map using the channel attention structure to obtain a channel weight feature map includes: Performing a maximum pooling operation and an average pooling operation on the third feature map to obtain a fourth sub-feature map and a fifth sub-feature map; Convolving the fourth sub-feature map to obtain a first single-channel map, and convolving the fifth sub-feature map to obtain a second single-channel map; splicing the first single-channel image and the second single-channel image to obtain a spliced single-channel image; Convolving the concatenated single-channel image to obtain a second single-channel feature map, and processing the second single-channel feature map with an activation function to obtain a second channel attention map; Perform a dot multiplication on the second channel attention map and the third feature map to obtain the channel weight feature map.

9. The classification method according to claim 5, characterized in that: The step of fusing the spatial weight feature map, the fusion weight feature map, and the channel weight feature map using the fusion structure in the multi-scale attention feature fusion structure to obtain the target weight feature map includes: Downsampling the spatial weight feature map to obtain a downsampled spatial weight feature map having the same scale as the fused weight feature map; Upsampling the channel weight feature map to obtain an upsampled channel weight feature map having the same scale as the fusion weight feature map; The fusion structure is used to fuse the downsampled spatial weight feature map, the fusion weight feature map and the upsampled channel weight feature map to obtain the target weight feature map.

10. A bill image classification device, characterized in that: include: The first processing unit is used to pre-process the bill image to obtain a target bill image; a second processing unit, configured to process the target bill image using a preset feature processing model to obtain a target weighted feature map, wherein the preset feature processing model at least includes: a multi-scale attention feature fusion structure; the multi-scale attention feature fusion structure is configured to extract feature maps of the target bill image at different scales, and weight the feature maps at different scales to obtain the target weighted feature map; The classification unit is used to classify the target weight feature map using a preset classifier to obtain the image category of the bill image.

Images