Image processing method, apparatus including image processing model, image processing apparatus, device, storage medium, and program product

Abstract

The invention provides an image processing method, a device comprising an image processing model, an image processing device, equipment, a storage medium and a program product, which can be applied to the technical field of artificial intelligence and the field of financial science and technology. The image processing method comprises: inputting a service image into a feature extraction network of an image processing model to obtain a plurality of feature maps, the feature extraction network comprising a sub-network and a target sub-network having the same structure, a target attention layer of the target sub-network and an attention layer of the sub-network having different types of attention mechanisms, the plurality of feature maps are obtained according to a first type of feature map and a second type of feature map, and the first type of feature map is obtained by inputting an output image of a target convolutional layer of a target sub-network into an attention layer of the sub-network; the second type of feature map is obtained by inputting the output image of the convolutional layer of the sub-network into the target attention layer of the target sub-network; and inputting the plurality of feature maps into an output layer of the image processing model to obtain an image classification result of the business image.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology and financial technology, and more specifically to an image processing method, a device including an image processing model, an image processing device, equipment, a storage medium, and a program product. Background Art

[0002] With the development of financial technology, business processes are complicated and cumbersome, and a large amount of information is required, which greatly increases the workload of archiving massive business information. The archiving process uses intelligent algorithms to classify the scanned massive business information.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: the feature map information extracted by the intelligent algorithm is relatively simple and it is difficult to capture the complex relationships in the image, resulting in inaccurate image classification. Summary of the invention

[0004] In view of the above problems, the present disclosure provides an image processing method, a device including an image processing model, an image processing device, an apparatus, a storage medium and a program product.

[0005] According to a first aspect of the present disclosure, an image processing method is provided, including: inputting a business image into a feature extraction network of an image processing model to obtain a plurality of feature maps, wherein the feature extraction network includes a subnetwork and a target subnetwork with the same structure, a target attention layer of the target subnetwork and an attention layer of the subnetwork have different types of attention mechanisms, the plurality of feature maps are obtained based on a first type of feature map and a second type of feature map, the first type of feature map is obtained by inputting an output image of a target convolutional layer of the target subnetwork into the attention layer of the subnetwork, and the second type of feature map is obtained by inputting an output image of the convolutional layer of the subnetwork into the target attention layer of the target subnetwork; the plurality of feature maps are input into the output layer of the image processing model to obtain an image classification result of the business image.

[0006] According to an embodiment of the present disclosure, inputting a business image into a feature extraction network of an image processing model to obtain a plurality of feature maps includes: inputting the business image into the i-th convolution layer of a subnetwork and the j-th target convolution layer of a target subnetwork, respectively, to obtain the i-th convolution feature map corresponding to the subnetwork and the j-th target convolution feature map corresponding to the target subnetwork, where i and j are both integers greater than 1 or equal to 1; inputting the i-th convolution feature map and the target convolution feature map into the i-th attention layer of the subnetwork to obtain a first type of feature map, where the target convolution feature is the image output by the target convolution layer of the target subnetwork; inputting the j-th target convolution feature map and the convolution feature map into the j-th target attention layer of the target subnetwork to obtain a second type of feature map, where the convolution feature map is the image output by the convolution layer of the subnetwork. Based on the first type of feature map and the second type of feature map, a plurality of feature maps are obtained.

[0007] According to an embodiment of the present disclosure, the business image is respectively input into the i-th convolutional layer of the subnetwork and the j-th target convolutional layer of the target subnetwork to obtain the i-th convolutional feature map corresponding to the subnetwork and the j-th target convolutional feature map corresponding to the target subnetwork, including: inputting the i-1-th attention map into the i-th convolutional layer of the subnetwork to obtain the i-th convolutional feature map corresponding to the subnetwork, the i-1-th attention map is obtained by inputting the i-1-th convolutional feature map and the target convolutional feature map into the i-1-th attention layer of the subnetwork, and the i-1-th convolutional feature map is obtained by inputting the business image into the i-1-th convolutional layer of the subnetwork, and i is an integer greater than 1.

[0008] According to an embodiment of the present disclosure, when i is equal to j, the sub-network and the target sub-network have different types of network parameters.

[0009] According to an embodiment of the present disclosure, inputting the i-th convolutional feature map and the target convolutional feature map into the i-th attention layer of the sub-network to obtain a first type of feature map also includes: performing an operation on the i-th convolutional feature map and the i-th target convolutional feature map to obtain an operation operation feature map; and inputting the operation operation feature map into the i-th attention layer of the sub-network to obtain a first type of feature map.

[0010] According to an embodiment of the present disclosure, the attention mechanisms of the target sub-network and the sub-network are a channel attention mechanism and a spatial attention mechanism, respectively.

[0011] According to an embodiment of the present disclosure, the method further includes: performing text recognition on the business image to obtain an identification of the business image; and managing the business image according to the image classification result and the identification of the business image.

[0012] A second aspect of the present disclosure provides a device including an image processing model, wherein the image processing model includes: a feature extraction network, which is used to extract features from an input business image to obtain multiple feature images, the feature extraction network includes a subnetwork and a target subnetwork with the same structure, the target attention layer of the target subnetwork and the attention layer of the subnetwork have different types of attention mechanisms, multiple feature maps are obtained based on first-class features and second-class feature maps, the first-class feature map is obtained by inputting the output image of the target convolution layer of the target subnetwork into the attention layer of the subnetwork, and the second-class feature map is obtained by inputting the output image of the convolution layer of the subnetwork into the target attention layer of the target subnetwork; an output layer, which is used to fuse the multiple feature images and output the image classification result of the business image.

[0013] The third aspect of the present disclosure provides an image processing device, including: a first input module, used to input a business image into a feature extraction network of an image processing model to obtain multiple feature maps, wherein the feature extraction network includes a subnetwork and a target subnetwork with the same structure, and the target attention layer of the target subnetwork and the attention layer of the subnetwork have different types of attention mechanisms, and the multiple feature maps are obtained according to the first type of features and the second type of feature maps, the first type of feature map is obtained by inputting the output image of the target convolution layer of the target subnetwork into the attention layer of the subnetwork, and the second type of feature map is obtained by inputting the output image of the convolution layer of the subnetwork into the target attention layer of the target subnetwork; the second input module is used to input the multiple feature maps into the output layer of the image processing model to obtain the image classification result of the business image.

[0014] A fourth aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] The fifth aspect of the present disclosure further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0016] The sixth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.

[0017] According to an embodiment of the present disclosure, a business image is input into a feature extraction network of an image processing model to obtain a plurality of feature maps, and the plurality of feature maps are input into the output layer of the image processing model to obtain an image classification result of the business image. The feature extraction network includes a subnetwork and a target subnetwork with the same structure, and different attention mechanisms are provided between the subnetworks and the target subnetworks. Since the plurality of feature maps are obtained based on the first type of feature maps and the second type of feature maps, the first type of feature maps are obtained by inputting the output image of the target convolution layer of the target subnetwork into the attention layer of the subnetwork, and the second type of feature maps are obtained by inputting the output image of the convolution layer of the subnetwork into the target attention layer of the target subnetwork, and the cross-use of different attention mechanisms between the subnetworks and the target subnetworks to extract the background and texture information of the business image can be achieved, thereby capturing the complex relationship of the business image, better integrating and extracting the feature map information outputted between the network layers, and improving the accuracy of image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0019] Figure 1 The application scenario diagram of the image processing method according to the embodiment of the present disclosure is schematically shown;

[0020] Figure 2a The flowchart of the image processing method according to the embodiment of the present disclosure is schematically shown;

[0021] Figure 2b A schematic diagram schematically shows an image processing model according to an embodiment of the present disclosure;

[0022] Figure 3 A flowchart of an image processing method according to another embodiment of the present disclosure is schematically shown;

[0023] Figure 4 A schematic diagram of a sub-network according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 5 A schematic diagram of a channel attention mechanism according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 6 A schematic diagram of a spatial attention mechanism according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 7a A schematic diagram of a feature extraction network according to an embodiment of the present disclosure is schematically shown;

[0027] Figure 7b A schematic diagram of a feature extraction network according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 7c A schematic diagram of a feature extraction network according to another embodiment of the present disclosure is schematically shown;

[0029] Figure 7d A schematic diagram of a feature extraction network according to another embodiment of the present disclosure is schematically shown;

[0030] Figure 8 The structure block diagram of the image processing device according to the embodiment of the present disclosure is schematically shown;

[0031] Fig. 9 A block diagram of an electronic device suitable for implementing an image processing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0033] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0034] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0035] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0036] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0037] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0038] An embodiment of the present disclosure provides an image processing method, including: inputting a business image into a feature extraction network of an image processing model to obtain multiple feature maps, wherein the feature extraction network includes a subnetwork and a target subnetwork with the same structure, the subnetwork and the target subnetwork both include a convolution layer and an attention layer, the attention layer of the target subnetwork and the attention layer of the subnetwork have different types of attention mechanisms, the multiple feature maps are obtained based on a first type of feature map and a second type of feature map, the first type of feature map is obtained by inputting an output image of a convolution layer of a target subnetwork into the attention layer of the subnetwork, and the second type of feature map is obtained by inputting an output image of a convolution layer of a subnetwork into the attention layer of the target subnetwork; the multiple feature maps are input into the output layer of the image processing model to obtain an image classification result of the business image.

[0039] Figure 1 The application scenario diagram of the image processing method according to the embodiment of the present disclosure is schematically shown.

[0040] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0041] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0042] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0043] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0044] It should be noted that the image processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the image processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The image processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the image processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0045] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0046] Figure 2a The flowchart of the image processing method according to the embodiment of the present disclosure is schematically shown.

[0047] like Figure 2a As shown, the image processing method of this embodiment includes operations S210 to S220.

[0048] In operation S210, the business image is input into the feature extraction network of the image processing model to obtain multiple feature maps, wherein the feature extraction network includes a subnetwork and a target subnetwork with the same structure, the target attention layer of the target subnetwork and the attention layer of the subnetwork have different types of attention mechanisms, and the multiple feature maps are obtained based on the first type of feature maps and the second type of feature maps, the first type of feature map is obtained by inputting the output image of the target convolution layer of the target subnetwork into the attention layer of the subnetwork, and the second type of feature map is obtained by inputting the output image of the convolution layer of the subnetwork into the target attention layer of the target subnetwork.

[0049] According to the embodiments of the present disclosure, the image processing model can be used to first extract features from the business image to obtain a plurality of feature images, and then classify the business image using the feature images.

[0050] According to an embodiment of the present disclosure, the skeleton network of the sub-network and the target sub-network may be an Xception network, a reinforcement learning network, a generative adversarial network, a Transformer network, etc.

[0051] According to the embodiments of the present disclosure, the sub-network has the same structure as the target sub-network, that is, the number of convolutional layers and the number of attention layers of the sub-network and the target sub-network are the same. It should be noted that the types of network parameters of the sub-network and the target sub-network may be different. For example, network parameters such as convolution kernels, activation function parameters, and pooling layer parameters are different between the network and the target sub-network.

[0052] According to an embodiment of the present disclosure, since the target attention layer of the target subnetwork and the attention layer of the subnetwork have different types of attention mechanisms, the output image of the target convolution layer of the target subnetwork is input into the attention layer of the subnetwork to obtain a first type of feature map, and the output image of the convolution layer of the subnetwork is input into the target attention layer of the target subnetwork to obtain a second type of feature image, thereby realizing that the features extracted by the two subnetworks can be fused in the subnetwork or the target subnetwork to capture the complex relationships in the business image.

[0053] According to an embodiment of the present disclosure, the first type of feature map is continuously input into other convolutional layers of the subnetwork to extract higher-level features until reaching the last layer of the subnetwork, and outputting a feature map. The second type of feature map is continuously input into other convolutional layers of the target subnetwork to extract higher-level features until reaching the last layer of the target subnetwork, and outputting another feature map.

[0054] In operation S220, the plurality of feature maps are input into an output layer of an image processing model to obtain an image classification result of the business image.

[0055] According to an embodiment of the present disclosure, the output layer may be a fully connected layer, which is used to map the sub-network and the target sub-network output multiple feature maps to obtain an image classification result.

[0056] According to an embodiment of the present disclosure, the image classification result represents the management category to which the business image belongs. For example, according to the business category classification, the image classification result may be a Class A business. According to the business settlement method classification, the image classification result may be a public settlement business.

[0057] Figure 2b A schematic diagram of an image processing model according to an embodiment of the present disclosure is schematically shown.

[0058] like Figure 2b As shown, the image processing model 200 of this embodiment includes a feature extraction network 210 and an output layer 220. The feature extraction network 210 includes a sub-network 211 and a target sub-network 212 with the same structure. The sub-network 211 includes a convolution layer and an attention layer, and the target sub-network 212 includes a target convolution layer and a target attention layer. The target attention layer of the target sub-network has a different type of attention mechanism from the attention layer of the sub-network.

[0059] According to an embodiment of the present disclosure, the sub-network 211 and the target sub-network 212 with the same structure have the same number of attention layers and convolution layers, and the distribution of the attention layers and the convolution layers in the network is also the same.

[0060] The feature extraction network 210 is used to extract features from the input business image to obtain multiple feature images. The multiple feature maps are obtained based on the first type of features and the second type of feature maps. The first type of feature map is obtained by inputting the output image of the target convolution layer of the target subnetwork 212 into the attention layer of the subnetwork 211, and the second type of feature map is obtained by inputting the output image of the convolution layer of the subnetwork 211 into the target attention layer of the target subnetwork 212.

[0061] The output layer 220 is used to fuse multiple feature images and output the image classification result of the business image.

[0062] According to an embodiment of the present disclosure, the parameters of the convolution layer of the subnetwork and the target convolution layer of the target subnetwork may be the same or different.

[0063] For example, when the convolution layer of the subnetwork has the same parameters as the target convolution layer of the target subnetwork, the output image of the 1st convolution layer of the subnetwork is input into any of the 2nd, 3rd, 4th, etc. (i.e., the attention layers except the 1st target attention layer of the target subnetwork) of the target subnetwork, so as to cross-capture the business image features.

[0064] Exemplarily, when the parameters of the convolution layer of the subnetwork are the same as those of the target convolution layer of the target subnetwork, the output image of the 1st convolution layer of the subnetwork is fused with the output images of the 2nd, 3rd, 4th, etc. target convolution layers of the target subnetwork (i.e., the target convolution layers except the 1st target convolution layer of the target subnetwork), and then respectively input into the 2nd, 3rd, 4th, etc. target attention layers of the target subnetwork, so that the low-level features of the subnetwork can be fused with the high-level features of the target subnetwork, thereby capturing more complex relationships of the business image.

[0065] Exemplarily, when the parameters of the convolution layer of the subnetwork are different from those of the target convolution layer of the target subnetwork, the output image of the first convolution layer of the subnetwork is input into any target attention layer of the target subnetwork.

[0066] For example, when the parameters of the convolution layer of the subnetwork and the target convolution layer of the target subnetwork are different, the output image of the first convolution layer of the subnetwork is fused with the output image of any target convolution layer of the target subnetwork, and then respectively input into any target attention layer of the target subnetwork.

[0067] According to an embodiment of the present disclosure, a business image is input into a feature extraction network of an image processing model to obtain a plurality of feature maps, and the plurality of feature maps are input into the output layer of the image processing model to obtain an image classification result of the business image. The feature extraction network includes a subnetwork and a target subnetwork with the same structure, and different attention mechanisms are provided between the subnetworks and the target subnetworks. Since the plurality of feature maps are obtained based on the first type of feature maps and the second type of feature maps, the first type of feature maps are obtained by inputting the output image of the target convolution layer of the target subnetwork into the attention layer of the subnetwork, and the second type of feature maps are obtained by inputting the output image of the convolution layer of the subnetwork into the target attention layer of the target subnetwork, and the cross-use of different attention mechanisms between the subnetworks and the target subnetworks to extract the background and texture information of the business image can be achieved, thereby capturing the complex relationship of the business image, better integrating and extracting the feature map information outputted between the network layers, and improving the accuracy of image classification.

[0068] According to an embodiment of the present disclosure, inputting a business image into a feature extraction network of an image processing model to obtain a plurality of feature maps includes: inputting the business image into the i-th convolution layer of a subnetwork and the j-th target convolution layer of a target subnetwork, respectively, to obtain the i-th convolution feature map corresponding to the subnetwork and the j-th target convolution feature map corresponding to the target subnetwork, where i and j are both integers greater than 1 or equal to 1; inputting the i-th convolution feature map and the target convolution feature map into the i-th attention layer of the subnetwork to obtain a first type of feature map, where the target convolution feature map is an image output by the target convolution layer of the target subnetwork; inputting the j-th target convolution feature map and the convolution feature map into the j-th target attention layer of the target subnetwork to obtain a second type of feature map, where the convolution feature map is an image output by the convolution layer of the subnetwork. Based on the first type of feature map and the second type of feature map, a plurality of feature maps are obtained.

[0069] According to an embodiment of the present disclosure, when the parameters of the convolution layer of the subnetwork are different from those of the target convolution layer of the target subnetwork, the target convolution feature map may be an image output by any target convolution layer of the target subnetwork.

[0070] According to an embodiment of the present disclosure, when the parameters of the convolution layer of the sub-network are different from the parameters of the target convolution layer of the target sub-network, the convolution feature map is an image output by any convolution layer of the sub-network.

[0071] According to an embodiment of the present disclosure, the sub-network and the target sub-network use different attention mechanisms crosswise to obtain the first type of feature map and the second type of feature map, so that the features of the background and texture of the business image are refined and the complex relationship of the business image can be captured.

[0072] According to an embodiment of the present disclosure, the business image is respectively input into the i-th convolutional layer of the subnetwork and the j-th target convolutional layer of the target subnetwork to obtain the i-th convolutional feature map corresponding to the subnetwork and the j-th target convolutional feature map corresponding to the target subnetwork, including: inputting the i-1-th attention map into the i-th convolutional layer of the subnetwork to obtain the i-th convolutional feature map corresponding to the subnetwork, the i-1-th attention map is obtained by inputting the i-1-th convolutional feature map and the target convolutional feature map into the i-1-th attention layer of the subnetwork, and the i-1-th convolutional feature map is obtained by inputting the business image into the i-1-th convolutional layer of the subnetwork, and i is an integer greater than 1.

[0073] According to an embodiment of the present disclosure, the network layers in the structure of the subnetwork may be "i-1 convolution layer, i-1 attention layer, i convolution layer, i attention layer" in sequence, that is, the i-1th attention map (also the i-1th first-class feature map) is input into the i-th convolution layer of the subnetwork to obtain the i-th convolution feature map corresponding to the subnetwork, and the i-th convolution feature map and the target convolution feature map are input into the i-th attention layer of the subnetwork to obtain the i-th first-class feature map.

[0074] According to an embodiment of the present disclosure, the network layers of the subnetwork and the target subnetwork are "i-1 convolution layer, i-1 attention layer, i convolution layer, i attention layer" respectively. The subnetwork and the target subnetwork use different attention mechanisms alternately, which can integrate the convolution features of different levels and the features of different attention mechanisms in the subnetwork and the target subnetwork to capture the complex relationship of the business image.

[0075] According to an embodiment of the present disclosure, when i is equal to j, the sub-network and the target sub-network have different types of network parameters.

[0076] According to an embodiment of the present disclosure, i is equal to j, and the sub-network and the target sub-network have different types of network parameters, that is, the sub-network and the target sub-network have parallel processing characteristics.

[0077] According to the embodiments of the present disclosure, the sub-network and the target sub-network process features in parallel, which not only speeds up the training of the image processing model but also speeds up the inference process of the model.

[0078] According to an embodiment of the present disclosure, inputting the i-th convolutional feature map and the target convolutional feature map into the i-th attention layer of the sub-network to obtain a first type of feature map also includes: performing an operation on the i-th convolutional feature map and the i-th target convolutional feature map to obtain an operation operation feature map; and inputting the operation operation feature map into the i-th attention layer of the sub-network to obtain a first type of feature map.

[0079] According to an embodiment of the present disclosure, the arithmetic operation may be image addition, subtraction, multiplication, division, threshold operation, image sharpening, etc.

[0080] For example, an addition operation is performed on the i-th convolution feature map and the i-th target convolution feature map to obtain an operation operation feature map.

[0081] According to an embodiment of the present disclosure, without affecting feature changes, the i-th convolution feature map and the i-th target convolution feature map are combined into an operation feature map to improve the efficiency of calculation and model training.

[0082] According to an embodiment of the present disclosure, the attention mechanisms of the target sub-network and the sub-network are a channel attention mechanism and a spatial attention mechanism, respectively.

[0083] According to an embodiment of the present disclosure, the channel attention mechanism may be SE (Squeeze-and-Excitation, compression and excitation attention mechanism), ECA (Efficient Channel Attention, efficient channel attention mechanism), CBAM (Convolutional Block Attention Module, convolutional block attention mechanism), etc.

[0084] According to an embodiment of the present disclosure, the spatial attention mechanism may be SA (Spatial Attention), CBAM (Convolutional Block Attention Module), BAM (Bottleneck Attention Module), and the like.

[0085] According to an embodiment of the present disclosure, the channel attention mechanism and the spatial attention mechanism are cross-used to extract the background and texture information of the business image, refine the feature map, and improve the accuracy of classification.

[0086] According to an embodiment of the present disclosure, the method further includes: performing text recognition on the business image to obtain an identification of the business image; and managing the business image according to the image classification result and the identification of the business image.

[0087] For example, optical character recognition (OCR) can be used to perform text recognition on business images.

[0088] For example, a trained recognition model can be used to perform text recognition on business images. The recognition model is obtained by training historical business images. The recognition model can be a support vector machine, a neural network model, a random forest algorithm, etc.

[0089] For example, a text recognition network layer can be set in the image processing model so that business image classification and text recognition are integrated into one model.

[0090] According to an embodiment of the present disclosure, the identifier of the business image represents a unique code of the image. For example, the identifier of the business image may be Y***8.

[0091] For example, according to the image classification result of the business image, the business image can be classified into a target category. Under the target category, the identifier of the business image is registered to implement classification management of the business image.

[0092] According to the embodiments of the present disclosure, text recognition is performed on a business image to obtain an identifier of the business image, thereby improving the management efficiency of the business image.

[0093] Figure 3 The flowchart of an image processing method according to another embodiment of the present disclosure is schematically shown.

[0094] like Figure 3 As shown, the image processing method of this embodiment includes operations S310 to S380.

[0095] In operation S310, the business image is input into the i-th convolutional layer of the subnetwork and the i-th target convolutional layer of the target subnetwork respectively to obtain the i-th convolutional feature map corresponding to the subnetwork and the i-th target convolutional feature map corresponding to the target subnetwork, where i is an integer greater than 1 or equal to 1.

[0096] In operation S320, an operation is performed on the i-th convolution feature map and the i-th target convolution feature map to obtain an operation-operated feature map.

[0097] In operation S330, the operation feature map is input into the i-th attention layer of the sub-network to obtain a first type of feature map.

[0098] In operation S340, the operation feature map is input into the i-th target attention layer of the target sub-network to obtain a second type of feature map.

[0099] In operation S350, a plurality of feature maps are obtained according to the first type of feature map and the second type of feature map.

[0100] In operation S360, the plurality of feature maps are input into an output layer of the image processing model to obtain an image classification result of the business image.

[0101] In operation S370, text recognition is performed on the business image to obtain an identifier of the business image.

[0102] In operation S380, the business image is managed according to the image classification result and the identification of the business image.

[0103] Regarding the management and classification of business images, the usual archiving method is to archive them directly after scanning, without automatic verification and classified storage, making it difficult to discover errors in business handling in a timely manner. At the same time, it takes a long time to retrieve business images, which is not convenient for management after archiving.

[0104] According to an embodiment of the present disclosure, the image processing model can be a parallel multi-cross convolutional neural network architecture to enhance the expressive power of the model so that it can better capture the complex relationship of the input data. The target subnetwork and the subnetwork have different attention mechanisms, which respectively aggregate the extracted feature maps of the deep feature attention from the channel and spatial dimensions, and refine the attention map of the neural network, in order to better integrate and extract the output feature map information between modules. OCR technology is introduced to verify the standardization, integrity and closed loop of business images.

[0105] The image processing model combines two neural networks with exactly the same structure, with the feature map size as the dividing point. The feature maps between each network layer or module are cross-fused to handle complex tasks and improve the performance of the model. It can enhance the expressive power of the image processing model and enable it to better capture the complex relationships of business images.

[0106] OCR systems usually include multiple processing steps, such as text detection, text segmentation, and character recognition. They have certain limitations in dealing with complex backgrounds, handwritten text, and diverse texts. These defects limit their performance and reliability in certain application scenarios. The introduction of image processing models has brought significant improvements to OCR, allowing the construction of end-to-end OCR models, and learning text information directly from business images through neural network structures, simplifying the entire system process and improving efficiency and accuracy.

[0107] The image processing model includes a feature extraction network and an output layer. The feature extraction network includes two sub-networks and a target sub-network with exactly the same structure, but their network parameters are different. The same business image is input into the two networks respectively. According to the size of the feature map, the feature maps of the same size in the two parallel networks are added and then cross-channel and spatial attention mechanisms are used to extract the background and texture information of the voucher image. Finally, the classification result is output through the output layer (such as the Dense layer).

[0108] Figure 4 The diagram schematically shows a sub-network according to an embodiment of the present disclosure.

[0109] like Figure 4 As shown in the figure, the sub-network can be an Xception network. Depthwise Separable Convolution is used to replace the original convolution operation to capture more features and have a deeper model depth, so that more features in the business image can be learned. In addition, Xception uses a very effective residual connection, which helps to avoid gradient disappearance and accelerate training, improves the effect of the training model without basically increasing the complexity of the network, and has better performance on larger data sets.

[0110] It should be noted that an attention layer can be set in the first convolution layer 402, the second convolution layer 403, the first separable convolution layer 404, the third convolution layer 405, the second separable convolution layer 406, the fourth convolution layer 407, the third separable convolution layer 408, the fifth convolution layer 409, the fourth separable convolution layer 410, the fifth separable convolution layer 411, the sixth convolution layer 412 and the sixth separable convolution layer 413 of the sub-network. In the sub-network, the business image 401 is input into the first convolution layer 402 to obtain the first convolution feature map, and then the first convolution feature map is input into the first attention layer until the output image of the sixth separable convolution layer 413 is input into the fully connected layer 413.

[0111] The target sub-network has the same structure as the sub-network, but the activation functions, strides, and convolutional layer parameters of each convolutional layer are different.

[0112] The feature extraction network is composed of channel and spatial attention mechanisms. Combining these two attention mechanisms, it can learn the attention mapping of feature maps in channels and spaces, and multiply it with the input to generate refined feature maps.

[0113] Figure 5 A schematic diagram of a channel attention mechanism according to an embodiment of the present disclosure is schematically shown.

[0114] like Figure 5 As shown in the figure, the channel attention mechanism can be Global Channel Efficient Attention (GCEA), which further improves the capture of detail information and uses global maximum pooling to explore the prominent details in the input features, supplementing the average pooling features.

[0115] Input convolution features are converted into vectors using global average pooling and global maximum pooling respectively and , H, W, and C represent the length, width, and number of channels of the feature map, respectively. The mathematical formula is as follows:

[0116] (1)

[0117] The max() function is used to extract the maximum value of the feature map channel. Next, this paper will vector V A and V M Convert to a new vector in the channel dimension and , concatenate them in the third dimension, and then use one-dimensional convolution to extract the channel attention mapping relationship.

[0118] (2)

[0119] σ represents the Sigmoid activation function, Conv1D( ) represents the one-dimensional convolution operation, Represents the extracted channel attention mapping weight, and then V att Convert the second dimension to the third dimension to form a vector , and then multiply it element by element with the input feature to get the channel attention mapping relationship In ECA (Efficient Channel Attention), local cross-channel interactions are captured by using one-dimensional convolution with adaptively selected convolution kernel size. The convolution kernel size is calculated as follows:

[0120] (3)

[0121] | | odd The symbol indicates that its value is taken as the nearest odd number. The EGCA parameter is twice that of the ECA, but since the original ECA parameter is generally 3, 5 or 7, the complexity of the GECA does not increase significantly.

[0122] Figure 6 A schematic diagram of a spatial attention mechanism according to an embodiment of the present disclosure is schematically shown.

[0123] According to an embodiment of the present disclosure, the spatial attention mechanism can be a multi-level spatial aggregation of attention, which extracts spatial features of different scales by performing convolutions at multiple scales; at the same time, superimposing multiple convolutions helps to extract and combine fine spatial information, thereby enriching the extracted spatial fine-grained information.

[0124] By performing average pooling and maximum pooling operations on the input feature F' in the channel dimension, the position matrix is ​​obtained and generated and The two position matrices are used to extract 3×3, 5×5 and 7×7 receptive field feature maps by using three 3×3 convolution kernels, which reduces the amount of computation while performing convolution at multiple scales; they are then connected in series to extract the attention map, and finally the outputs of the three convolutions are cascaded.

[0125] (4)

[0126] In the above formula (4) , and Represents the spatial attention mapping matrices extracted by 3×3, 5×5 and 7×7 respectively; Represents a 1×1 convolution, which is used to integrate the spatial attention maps extracted by these three convolutions, and finally the integrated attention matrix M att Multiplying with the input feature map F' will eventually multiply with to obtain the spatial attention feature map .

[0127] Perform OCR recognition on the classified business images, call the OCR recognition API interface, perform content recognition on the classified business images to obtain identification, use the identification for effective archiving and storage, strengthen credential management, and increase the credential image access rate. Improving the retrieval rate of credential images while achieving classified permanent storage is helpful for organizational management.

[0128] Figure 7a A schematic diagram of a feature extraction network according to an embodiment of the present disclosure is schematically shown.

[0129] like Figure 7a As shown, the feature extraction network 700 of this embodiment includes a sub-network 710 and a target sub-network 720. The sub-network 710 has multiple convolutional layers and multiple attention layers; the target sub-network 720 has multiple target convolutional layers and multiple target attention layers.

[0130] i and j are both integers greater than 1 or equal to 1, and i is greater than j. The business image is input into the i-th convolution layer of the subnetwork 710 and the j-th target convolution layer of the target subnetwork 720, respectively, to obtain the i-th convolution feature map corresponding to the subnetwork 710 and the j-th target convolution feature map corresponding to the target subnetwork 720; the i-th convolution feature map and the target convolution feature map are input into the i-th attention layer of the subnetwork 710 to obtain a first-class feature map, where the target convolution feature is an image output by any target convolution layer of the target subnetwork 720, for example, the target convolution feature map is an image output by the first target convolution layer of the target subnetwork 720 or an image output by the j-th target convolution layer.

[0131] The j-th target convolutional feature map and the convolutional feature map are input into the j-th target attention layer of the target sub-network 720 to obtain a second type of feature map, where the convolutional feature map is an image output by any convolutional layer of the sub-network 710, for example, the convolutional feature map is an image output by the first convolutional layer of the sub-network 720. According to the first type of feature map and the second type of feature map, a plurality of feature maps are obtained.

[0132] Figure 7b A schematic diagram of a feature extraction network according to an embodiment of the present disclosure is schematically shown.

[0133] like Figure 7b As shown, the feature extraction network 700 of this embodiment includes a subnetwork 710 and a target subnetwork 720. i and j are both integers greater than 1. The subnetwork 710 has a total of i convolutional layers and i-1 attention layers, and the target subnetwork 720 has i target convolutional layers and i-1 target attention layers.

[0134] The business image is input into the i-1th convolution layer of the subnetwork 710 and the i-1th target convolution layer of the target subnetwork 720 respectively, and the i-1th convolution feature map corresponding to the subnetwork 710 and the i-1th target convolution feature map corresponding to the target subnetwork 720 are obtained; the i-1th convolution feature map and the target convolution feature map are input into the i-1th attention layer of the subnetwork 710 to obtain a first type of feature map, and the target convolution feature is the image output by any target convolution layer of the target subnetwork 720; the i-1th target convolution feature map and the convolution feature map are input into the i-1th target attention layer of the target subnetwork 720 to obtain a second type of feature map, and the convolution feature map is the image output by any convolution layer of the subnetwork 710. The first type of feature map is input into the i-th convolution layer of the subnetwork 710 to obtain a feature map; the second type of feature map is input into the i-th target convolution layer of the target subnetwork 720 to obtain another feature map.

[0135] Figure 7c A schematic diagram of a feature extraction network according to another embodiment of the present disclosure is schematically shown.

[0136] like Figure 7c As shown, the feature extraction network 700 of this embodiment includes a sub-network 710 and a target sub-network 720. When i is equal to j, i is an integer greater than 1 or equal to 1. The sub-network 710 and the target sub-network 720 have different types of network parameters.

[0137] Sub-network 710 has i+1 convolutional layers; target sub-network 720 has i+1 target convolutional layers.

[0138] The business image is input into the i-th convolution layer of the sub-network 710 and the i-th convolution layer of the target sub-network 720 respectively, and the i-th convolution feature map corresponding to the sub-network 710 and the i-th target convolution feature map corresponding to the target sub-network 710 are obtained; the i-th convolution feature map and the i-th target convolution feature map are operated to obtain an operation feature map; the operation feature map is input into the i-th attention layer of the sub-network 710 and the i-th target attention layer of the target sub-network 720 respectively, and the first type of feature map and the second type of feature map are obtained. The first type of feature map is input into the i+1th convolution layer of the sub-network 710 to obtain a feature map; the second type of feature map is input into the i+1th target convolution layer of the target sub-network 720 to obtain another feature map.

[0139] Figure 7d A schematic diagram of a feature extraction network according to yet another embodiment of the present disclosure is schematically shown.

[0140] like Figure 7d As shown, the feature extraction network 700 of this embodiment includes a subnetwork 710 and a target subnetwork 720. When i is equal to j, i is an integer greater than 1. The subnetwork 710 and the target subnetwork 720 have different types of network parameters. The subnetwork 710 has a total of i+1 convolutional layers; the target subnetwork 720 has a total of i+1 target convolutional layers. The attention mechanisms of the target subnetwork 720 and the subnetwork 710 are channel attention mechanisms and spatial attention mechanisms, respectively.

[0141] The business image is respectively input into the first convolution layer of the subnetwork 710 and the first target convolution layer of the target subnetwork 720 to obtain a first convolution feature map corresponding to the subnetwork 710 and a first target convolution feature map corresponding to the target subnetwork 710; the first convolution feature map and the first target convolution feature map are operated to obtain a first operation feature map; the first operation feature map is respectively input into the first attention layer of the subnetwork 710 and the first target attention layer of the target subnetwork 720 to obtain a first first-category feature map and a first second-category feature map.

[0142] The first first-category feature map is input into the second convolutional layer of the subnetwork 710 to obtain a second convolutional feature map; the first second-category feature map is input into the second target convolutional layer of the target subnetwork 720 to obtain a second target convolutional feature map; the second convolutional feature map and the second target convolutional feature map are operated to obtain a second operation feature map; the second operation feature map is respectively input into the second attention layer of the subnetwork 710 and the second target attention layer of the target subnetwork 720 to obtain a second first-category feature map and a second second-category feature map.

[0143] Until the i-th first-category feature map is input into the i+1-th convolutional layer of the sub-network 710 to obtain a feature map; the i-th second-category feature map is input into the i+1-th target convolutional layer of the target sub-network 720 to obtain another feature map.

[0144] A device including an image processing model, the image processing model including: a feature extraction network, used to extract features from an input business image to obtain multiple feature images, the feature extraction network including a subnetwork and a target subnetwork with the same structure, the attention layer of the target subnetwork and the attention layer of the subnetwork have different types of attention mechanisms, multiple feature maps are obtained based on first-class features and second-class feature maps, the first-class feature map is obtained by inputting the output image of the target convolution layer of the target subnetwork into the attention layer of the subnetwork, and the second-class feature map is obtained by inputting the output image of the convolution layer of the subnetwork into the target attention layer of the target subnetwork; an output layer, used to fuse the multiple feature images and output the image classification result of the business image.

[0145] According to an embodiment of the present disclosure, the schematic diagram of the image processing model is as follows: Figure 2b It should be noted that Figure 7a~7d The feature extraction networks shown can all be used as embodiments of image processing models.

[0146] Figure 8 The structure block diagram of the image processing device according to the embodiment of the present disclosure is schematically shown.

[0147] like Figure 8 As shown, the image processing device 800 of this embodiment includes a first input module 810 and a second input module 820 .

[0148] The first input module 810 is used to input the business image into the feature extraction network of the image processing model to obtain multiple feature maps, wherein the feature extraction network includes a subnetwork and a target subnetwork with the same structure, the attention layer of the target subnetwork and the attention layer of the subnetwork have different types of attention mechanisms, and the multiple feature maps are obtained based on the first type of features and the second type of feature maps, the first type of feature map is obtained by inputting the output image of the target convolution layer of the target subnetwork into the attention layer of the subnetwork, and the second type of feature map is obtained by inputting the output image of the convolution layer of the subnetwork into the target attention layer of the target subnetwork. In one embodiment, the first input module 810 can be used to perform the operation S210 described above, which will not be repeated here.

[0149] The second input module 820 is used to input multiple feature maps into the output layer of the image processing model to obtain the image classification result of the business image. In one embodiment, the second input module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0150] According to an embodiment of the present disclosure, the first input module includes a first input submodule, a second input submodule, a third input submodule and an acquisition submodule. The first input submodule is used to input the business image into the i-th convolution layer of the subnetwork and the j-th target convolution layer of the target subnetwork, respectively, to obtain the i-th convolution feature map corresponding to the subnetwork and the j-th target convolution feature map corresponding to the target subnetwork, where i and j are both integers greater than 1 or equal to 1; the second input submodule is used to input the i-th convolution feature map and the target convolution feature map into the i-th attention layer of the subnetwork to obtain a first type of feature map, where the target convolution feature is the image output by the target convolution layer of the target subnetwork; the third input submodule is used to input the j-th target convolution feature map and the convolution feature map into the j-th target attention layer of the target subnetwork to obtain a second type of feature map, where the convolution feature map is the image output by the convolution layer of the subnetwork. The acquisition submodule is used to obtain multiple feature maps based on the first type of feature map and the second type of feature map.

[0151] According to an embodiment of the present disclosure, the first input submodule includes a first input unit. The first input unit is used to input the i-1th attention map into the i-th convolution layer of the subnetwork to obtain the i-th convolution feature map corresponding to the subnetwork, the i-1th attention map is obtained by inputting the i-1th convolution feature map and the target convolution feature map into the i-1th attention layer of the subnetwork, the i-1th convolution feature map is obtained by inputting the business image into the i-1th convolution layer of the subnetwork, and i is an integer greater than 1.

[0152] According to an embodiment of the present disclosure, when i is equal to j, the sub-network and the target sub-network have different types of network parameters.

[0153] According to an embodiment of the present disclosure, the second input submodule further includes an operation unit and a second input unit. The operation unit is used to perform an operation on the i-th convolution feature map and the i-th target convolution feature map to obtain an operation operation feature map; the second input unit is used to input the operation operation feature map into the i-th attention layer of the subnetwork to obtain a first type of feature map.

[0154] According to an embodiment of the present disclosure, the attention mechanisms of the target sub-network and the sub-network are a channel attention mechanism and a spatial attention mechanism, respectively.

[0155] According to an embodiment of the present disclosure, the above device further includes an identification module and a management module. The identification module is used to perform text recognition on the business image to obtain the identification of the business image; the management module is used to manage the business image according to the image classification result and identification of the business image.

[0156] According to an embodiment of the present disclosure, any multiple modules of the first input module 810 and the second input module 820 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first input module 810 and the second input module 820 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the first input module 810 and the second input module 820 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0157] Fig. 9 A block diagram of an electronic device suitable for implementing an image processing method according to an embodiment of the present disclosure is schematically shown.

[0158] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0159] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0160] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0161] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0162] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0163] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the image processing method provided by the embodiment of the present disclosure.

[0164] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 901. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0165] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0166] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0167] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0168] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0169] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0170] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An image processing method, characterized in that: The method comprises: Inputting the business image into the feature extraction network of the image processing model to obtain a plurality of feature maps, wherein the feature extraction network includes a subnetwork and a target subnetwork with the same structure, the target attention layer of the target subnetwork and the attention layer of the subnetwork have different types of attention mechanisms, and the plurality of feature maps are obtained according to a first type of feature map and a second type of feature map, wherein the first type of feature map is obtained by inputting an output image of a target convolution layer of the target subnetwork into the attention layer of the subnetwork, and the second type of feature map is obtained by inputting an output image of a convolution layer of the subnetwork into the target attention layer of the target subnetwork; The multiple feature maps are input into the output layer of the image processing model to obtain the image classification result of the business image.

2. The method according to claim 1, characterized in that The business image is input into the feature extraction network of the image processing model to obtain multiple feature graphs including: Input the business image into the i-th convolutional layer of the subnetwork and the j-th target convolutional layer of the target subnetwork respectively, and obtain the i-th convolutional feature map corresponding to the subnetwork and the j-th target convolutional feature map corresponding to the target subnetwork, where i and j are both integers greater than 1 or equal to 1; Inputting the i-th convolutional feature map and the target convolutional feature map into the i-th attention layer of the subnetwork to obtain a first type of feature map, wherein the target convolutional feature map is an image output by the target convolutional layer of the target subnetwork; The j-th target convolutional feature map and the convolutional feature map are input into the j-th target attention layer of the target sub-network to obtain a second type of feature map, where the convolutional feature map is an image output by the convolutional layer of the sub-network. The multiple feature maps are obtained according to the first type of feature maps and the second type of feature maps.

3. The method according to claim 2, characterized in that Inputting the business image into the i-th convolutional layer of the subnetwork and the j-th target convolutional layer of the target subnetwork respectively, and obtaining the i-th convolutional feature map corresponding to the subnetwork and the j-th target convolutional feature map corresponding to the target subnetwork includes: The i-1th attention map is input into the i-1th convolutional layer of the sub-network to obtain the i-1th convolutional feature map corresponding to the sub-network, wherein the i-1th attention map is obtained by inputting the i-1th convolutional feature map and the target convolutional feature map into the i-1th attention layer of the sub-network, and the i-1th convolutional feature map is obtained by inputting the business image into the i-1th convolutional layer of the sub-network, where i is an integer greater than 1.

4. The method according to claim 2, characterized in that: When i is equal to j, the sub-network and the target sub-network have different network parameter types.

5. The method according to claim 4, characterized in that The step of inputting the i-th convolutional feature map and the target convolutional feature map into the i-th attention layer of the subnetwork to obtain the first type of feature map further includes: Performing a computation operation on the i-th convolution feature map and the i-th target convolution feature map to obtain a computation operation feature map; The operation feature map is input into the i-th attention layer of the sub-network to obtain the first type of feature map.

6. The method according to claim 1, characterized in that The attention mechanisms of the target sub-network and the sub-network are respectively a channel attention mechanism and a spatial attention mechanism.

7. The method according to claim 1, characterized in that The method further comprises: Performing text recognition on the business image to obtain an identifier of the business image; The business image is managed according to the image classification result of the business image and the identifier.

8. A device comprising an image processing model, characterized in that The image processing model includes: A feature extraction network, used to extract features from an input business image to obtain multiple feature images, wherein the feature extraction network includes a subnetwork and a target subnetwork with the same structure, the target attention layer of the target subnetwork and the attention layer of the subnetwork have different types of attention mechanisms, the multiple feature maps are obtained according to the first type of features and the second type of feature maps, the first type of feature map is obtained by inputting the output image of the target convolution layer of the target subnetwork into the attention layer of the subnetwork, and the second type of feature map is obtained by inputting the output image of the convolution layer of the subnetwork into the target attention layer of the target subnetwork; The output layer is used to fuse the multiple feature images and output the image classification result of the business image.

9. An image processing device, characterized in that: The device comprises: A first input module is used to input a business image into a feature extraction network of an image processing model to obtain a plurality of feature maps, wherein the feature extraction network includes a subnetwork and a target subnetwork with the same structure, a target attention layer of the target subnetwork and an attention layer of the subnetwork have different types of attention mechanisms, and the plurality of feature maps are obtained according to first-class features and second-class feature maps, wherein the first-class feature map is obtained by inputting an output image of a target convolutional layer of the target subnetwork into the attention layer of the subnetwork, and the second-class feature map is obtained by inputting an output image of a convolutional layer of the subnetwork into the target attention layer of the target subnetwork; The second input module is used to input the multiple feature maps into the output layer of the image processing model to obtain the image classification result of the business image.

10. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images