An image processing method, apparatus, device and storage medium

Abstract

Embodiments of the present application provide an image processing method, device and equipment, and a storage medium, relating to the technical field of artificial intelligence. The method comprises the following steps: firstly, performing preliminary segmentation on a to-be-processed image to determine a region of interest containing a card face element; then, performing focus processing on the region of interest to obtain a core region of the card face element; then, based on the feature that the internal pixel distribution patterns of the same card face element are similar, constructing a pixel retrieval tree based on the core region; and finally, using the pixel retrieval tree to segment the region of interest to obtain a target region where the card face element is located, thereby improving the accuracy of card face element positioning and improving the positioning effect of the card face element. Furthermore, the present application does not require data labeling and model training in the process of positioning the card face element, thereby improving the efficiency of card face element positioning and reducing the cost. In addition, the method adopted by the present application has strong robustness and can still achieve good results under complex background conditions.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence, and in particular, to an image processing method and device, equipment and a storage medium. BACKGROUND

[0002] In actual applications, the size and position of the card surface elements of a bank card need to meet compliance requirements, but it is difficult for a human to judge such compliance. Therefore, a recognition device is needed to locate the card surface elements, and to automatically determine the compliance of the card surface elements according to the location result and business rules.

[0003] Related technologies use a deep neural network to perform image semantic segmentation on a bank card image to obtain the position of the card surface elements in the bank card image. However, the data used to train the deep neural network contains original images and corresponding semantic label images, where the semantic labels are obtained by manually classifying and labeling each pixel in the original image. Manual classification and labeling will inevitably result in labeling errors, and different people will also have inconsistent labeling results for the same image, thereby reducing the accuracy of the deep neural network in performing semantic segmentation on the bank card image, and further affecting the accuracy of the location of the card surface elements. SUMMARY

[0004] Embodiments of the present application provide an image processing method, device, equipment and storage medium, for improving the accuracy of card surface element location.

[0005] In one aspect, the present application provides an image processing method, which comprises:

[0006] obtaining a to-be-processed image;

[0007] performing preliminary segmentation on the to-be-processed image to determine a region of interest containing card surface elements;

[0008] focusing on processing the region of interest to obtain a core region of the card surface elements;

[0009] constructing a pixel retrieval tree based on the core region, and segmenting the region of interest using the pixel retrieval tree to obtain a target region where the card surface elements are located.

[0010] In one aspect, the present application provides an image processing device, which comprises:

[0011] an acquisition module configured to obtain a to-be-processed image;

[0012] an image segmentation module configured to perform preliminary segmentation on the to-be-processed image to determine a region of interest containing card surface elements;

[0013] a focusing module, configured to focus on the region of interest to obtain a core region of the card face element;

[0014] a positioning module, configured to construct a pixel search tree based on the core region, and segment the region of interest based on the pixel search tree to obtain a target region where the card face element is located.

[0015] Optionally, the focusing module is specifically configured to:

[0016] extract edge features in the region of interest;

[0017] cut out the core region of the card face element from the region of interest based on the edge features.

[0018] Optionally, the positioning module is specifically configured to:

[0019] take pixels in the core region as an initial seed pixel set;

[0020] iteratively update the seed pixel set until the seed pixel set meets an iteration stopping condition, and take the seed pixel set obtained when the iteration is stopped as the target region where the card face element is located, wherein each iteration process includes:

[0021] construct a pixel search tree based on a candidate pixel set adjacent to the seed pixel set;

[0022] acquire at least one matching connected domain from the region of interest based on the pixel search tree, and merge the at least one matching connected domain to the seed pixel set.

[0023] Optionally, the positioning module is specifically configured to:

[0024] retrieve a plurality of matching pixels from the region of interest that match the pixel search tree;

[0025] determine at least one pixel connected domain based on the plurality of matching pixels obtained;

[0026] select at least one matching connected domain from the at least one pixel connected domain based on an association relationship between the at least one pixel connected domain and the region of interest, respectively.

[0027] Optionally, the positioning module is specifically configured to:

[0028] convert a pixel value of each pixel in the region of interest into a string;

[0029] if the string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matching pixel.

[0030] Optionally, the positioning module is specifically configured to:

[0031] For each pixel connected domain, if the pixel connected domain has an overlapping area with the region of interest, and a ratio of an area of the overlapping area to an area of the pixel connected domain is greater than a preset threshold, the pixel connected domain is regarded as a matching connected domain.

[0032] Optionally, the seed pixel set satisfying the iteration stop condition means that the seed pixel set no longer changes.

[0033] Optionally, the focusing module is specifically configured to:

[0034] The length and the width of the region of interest are respectively focused on a preset value based on the unchanged center position of the region of interest, to obtain a core region of the card surface element.

[0035] Optionally, the positioning module is specifically configured to:

[0036] Based on the pixels in the core region, a pixel retrieval tree is constructed;

[0037] The region of interest is expanded to obtain an expansion result region;

[0038] Based on the pixel retrieval tree, at least one matching connected domain is obtained from the expansion result region, and the at least one matching connected domain is merged into the core region to obtain a target region where the card surface element is located.

[0039] Optionally, the positioning module is specifically configured to:

[0040] A plurality of matching pixels are retrieved from the expansion result region and matched with the pixel retrieval tree;

[0041] Based on the obtained plurality of matching pixels, at least one pixel connected domain is determined;

[0042] For each pixel connected domain, if an area of the pixel connected domain is greater than or equal to a preset threshold, the pixel connected domain is regarded as a matching connected domain.

[0043] Optionally, the positioning module is specifically configured to:

[0044] For each pixel in the expansion result region, a pixel value of the pixel is converted into a string; if a string corresponding to the pixel matches a node in the pixel retrieval tree, the pixel is a matching pixel.

[0045] Optionally, the image segmentation module is specifically configured to:

[0046] The deep neural network model is used to preliminarily segment the image to be processed to determine a region of interest containing the card face element.

[0047] Optionally, the image segmentation module is further configured to:

[0048] After the region of interest is segmented by using the pixel search tree to obtain the target region where the card face element is located, the image to be processed in which the target region is marked is used as a training sample to update the deep neural network model.

[0049] In an aspect, an embodiment of the present application provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the image processing method when executing the program.

[0050] In an aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program executable by a computer device, and when the program is executed on the computer device, the computer device executes the steps of the image processing method.

[0051] In the embodiment of the present application, the image to be processed is preliminarily segmented to determine a region of interest containing the card face element, and then the region of interest is focused to obtain a core region of the card face element, and then a pixel search tree is constructed based on the core region by using the similar characteristics of the internal pixel distribution mode of the same card face element, and the region of interest is segmented by using the pixel search tree to obtain a target region where the card face element is located, thereby improving the accuracy of the card face element positioning and improving the positioning effect of the card face element. Secondly, the present application does not need to perform data labeling and model training in the process of positioning the card face element, thereby improving the efficiency of the card face element positioning and reducing the cost. In addition, the method used in the present application has strong robustness and can still achieve good results under complex background conditions. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0053] Figure 1 A structural schematic diagram of a system architecture provided by an embodiment of the present application;

[0054] Figure 2 A flowchart of an image processing method provided by an embodiment of the present application Figure 1 ;

[0055] Figure 3 This is a schematic diagram of the structure of a deep neural network model provided in an embodiment of this application;

[0056] Figure 4 A flowchart illustrating an image processing method provided in this application embodiment. Figure 2 ;

[0057] Figure 5 A flowchart illustrating an image processing method provided in this application embodiment. Figure 3 ;

[0058] Figure 6 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0059] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0061] For ease of understanding, the terms used in the embodiments of this invention are explained below.

[0062] ROI: Region of Interest.

[0063] Faster R-CNN: Faster Region-CNN, a fast end-to-end object detection deep neural network.

[0064] YOLO: You Look Once, is a commonly used algorithm in object detection.

[0065] U-net: u-shaped network, U-shaped convolutional neural network.

[0066] FCN: Fully Convolutional Networks.

[0067] refer to Figure 1 This is a system architecture diagram applicable to the embodiments of this application. The system architecture includes at least terminal device 101 and server 102. The number of terminal devices 101 can be one or more, and the number of servers 102 can also be one or more. This application does not specifically limit the number of terminal devices 101 and servers 102.

[0068] The terminal device 101 pre-installs an application program for positioning the card face element, and the application program is a client application, a web application, a mini-program application, etc. The terminal device 101 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart home appliance, a smart voice interaction device, a smart vehicle device, etc., but is not limited thereto.

[0069] The server 102 is a background server of the application program, and the server 102 can be a stand-alone physical server, a server cluster composed of multiple physical servers, or a distributed system, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal device 101 and the server 102 can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0070] The image processing method in the embodiment of the present application can be executed by the terminal device 101 or the server 102, or can be executed by the terminal device 101 and the server 102 in interaction.

[0071] Based on the system architecture diagram shown in Figure 1 The embodiment of the present application provides a flow of an image processing method, as shown in Figure 2 The flow of the method is executed by a computer device, which can be Figure 1 The terminal device 101 and / or the server 102 shown in, including the following steps:

[0072] Step S201, obtaining a to-be-processed image.

[0073] Specifically, the to-be-processed image can be an image of a bank card, a shopping card, a membership card, etc. The to-be-processed image can be obtained by photographing, or can be a design drawing, which is not limited in the present application.

[0074] Step S202, performing preliminary segmentation on the to-be-processed image to determine a region of interest containing a card face element.

[0075] In some embodiments, a deep neural network model is used to perform preliminary segmentation on the to-be-processed image to determine a region of interest containing a card face element, and the deep neural network model can be a U-Net, an FCN, a Faster R-CNN, a yolo, etc.

[0076] In some embodiments, in addition to obtaining the region of interest containing the card face element by using the deep neural network model, the type of the card face element can also be obtained.

[0077] Taking the Faster R-CNN as an example of the deep neural network model, referring to Figure 3 , the deep neural network model 300 includes a convolutional layer 301, a region proposal network (RPN) 302, a target region pooling layer 303, and a classifier 304. The bank card image is input into the convolutional layer 301, and the convolutional layer 301 extracts the image features of the bank card image, and then shares the image features with the region proposal network 302 and the target region pooling layer 303. The region proposal network 302 generates a region candidate image block based on the image features, and realizes the positioning of the region of interest of the card face element. The target region pooling layer 303 extracts a feature map of the region of interest based on the image features. The classifier 304 outputs the type of the card face element and the region of interest based on the region candidate image block and the feature map of the region of interest.

[0078] In step S203, the region of interest is focused to obtain the core region of the card face element.

[0079] Specifically, focusing the region of interest refers to cropping the region of interest to remove the background region in the region of interest, or removing the background region and the peripheral region of the card face element in the region of interest, so as to obtain the core region of the card face element.

[0080] In some embodiments, the length and width of the region of interest are respectively focused to a preset value based on the unchanged center position of the region of interest, to obtain the core region of the card face element.

[0081] Specifically, when the card face element contains less color features, such as the chip identifier on the bank card, the length and width of the region of interest are respectively focused to a preset value to obtain the core region of the card face element.

[0082] For example, the bank card image is input into the deep neural network model, and the region of interest of the card face element output by the deep neural network model is ROI = [(xroimin, yroimin), (xroimax, yroimax)]. The length and width of the region of interest are respectively focused to a preset value a based on the unchanged center position of the region of interest, and the core region of the card face element obtained is ROIF = [(xroimin + a, yroimin + a), (xroimax - a, yroimax - a)], and all the pixels in the core region ROIF are card face elements.

[0083] In some embodiments, edge features in the region of interest are extracted; based on the edge features, a core region of the card surface element is cut out from the region of interest.

[0084] Specifically, when the card surface element contains more color features, such as a bank logo on a bank card, edge features E0 in the region of interest are extracted, and then edges in the edge features E0 that are connected to the outside of the region of interest are removed to obtain a core region E1 of the card surface element.

[0085] In step S204, a pixel retrieval tree is constructed based on the core region, and the region of interest is segmented using the pixel retrieval tree to obtain a target region where the card surface element is located.

[0086] Specifically, the pixel retrieval tree includes a plurality of nodes, and each node corresponds to a string. The region of interest is finely segmented using the pixel retrieval tree to obtain an accurate region where the card surface element is located.

[0087] In the embodiments of the present application, the image to be processed is first preliminarily segmented to determine a region of interest containing a card surface element; then the region of interest is focused to obtain a core region of the card surface element; then the pixel retrieval tree is constructed based on the core region by taking advantage of the similar pixel distribution pattern within the same card surface element; and the region of interest is segmented using the pixel retrieval tree to obtain a target region where the card surface element is located, thereby improving the accuracy of the card surface element positioning and improving the positioning effect of the card surface element. Secondly, the present application does not need data labeling and model training in the process of positioning the card surface element, thereby improving the efficiency of the card surface element positioning and reducing the cost. In addition, the method used in the present application has strong robustness and can still achieve good results under complex background conditions.

[0088] Optionally, in the above step S204, the pixel retrieval tree is constructed based on the pixels in the core region; then the region of interest is expanded to obtain an expanded result region; at least one matching connected domain is obtained from the expanded result region based on the pixel retrieval tree, and the at least one matching connected domain is merged into the core region to obtain a target region where the card surface element is located.

[0089] Specifically, the region of interest ROI is taken as the center to expand the region, and the expanded result region is ROIE = [(xroimin-b, yroimin-b), (xroimax+b, yroimax+b)], so that the card surface element is completely contained in the ROIE region.

[0090] A plurality of matching pixels matching the pixel search tree are retrieved from the extended result region. Then, based on the obtained plurality of matching pixels, at least one pixel connected domain is determined; for each pixel connected domain, if the area of the pixel connected domain is greater than or equal to a preset threshold, the pixel connected domain is taken as a matching connected domain.

[0091] Specifically, for each pixel in the extended result region, the pixel value of the pixel is converted into a string; if the string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matching pixel, and if the string corresponding to the pixel does not match any node in the pixel search tree, the pixel is a non-matching pixel.

[0092] In actual application, for each pixel in the extended result region, the pixel value of the pixel is set as [R, G, B], and the pixel value of the pixel is converted into a string str(R)+str(G)+str(B), where str represents converting a number into a string and retaining 3 significant digits, for example: R=12, str(R)="012", "+" is a string concatenation operation.

[0093] It is judged whether the string str(R)+str(G)+str(B) matches the pixel search tree PT, i.e., whether the node corresponding to the string str(R)+str(G)+str(B) in the pixel search tree PT is 1; if yes, the pixel matches a node in the pixel search tree, i.e., the pixel is a matching pixel, and the original pixel value is retained, otherwise the pixel is set to 0.

[0094] Based on the obtained plurality of matching pixels, at least one pixel connected domain can be determined, and the area of each pixel connected domain can be calculated. When the area of the pixel connected domain is less than a preset threshold smin, the pixels in the pixel connected domain are set to 0. Finally, the accurate pixel set composed of the retained non-0 pixel connected domains is merged into the core region to obtain the target region where the card face element is located; the coordinate range [(xmin, ymin), (xmax, ymax)] of the pixels in the accurate pixel set is taken as the position of the target region in the bank card image.

[0095] For example, referring to Figure 4 A flowchart of an image processing method provided by the embodiment of the present application is shown in the figure, the method is interactively executed by a card face element positioning module, an element segmentation module, a pixel search module and a contour positioning module in a server, and includes the following steps:

[0096] Step 401, the card face element positioning module reads a bank card image.

[0097] Step 402, the card face element positioning module locates the region of interest and the type of the card face element in the bank card image by using the Faster R-CNN.

[0098] In step 403, the element segmentation module performs auto-focusing on the region of interest to obtain a core region ROIF of the card element.

[0099] Specifically, when the card element is of a preset element type with less color features (such as a chip identifier), the step 403 and subsequent steps are performed, otherwise the subsequent steps are not performed.

[0100] In step 404, the pixel retrieval module constructs a pixel retrieval tree based on the pixels in the core region ROIF.

[0101] In step 405, the element segmentation module performs region expansion on the region of interest to obtain an expanded result region ROIE.

[0102] In step 406, the pixel retrieval module retrieves all pixels in the expanded result region ROIE in the pixel retrieval tree to obtain matching pixels in the expanded result region ROIE.

[0103] In step 407, the element segmentation module retains the pixel values of the matching pixels and sets the pixel values of the non-matching pixels to 0.

[0104] In step 408, the element segmentation module determines at least one pixel connected domain based on the obtained matching pixels.

[0105] In step 409, the element segmentation module sets the pixel values of the pixels in the pixel connected domain with an area smaller than a preset threshold smin to 0.

[0106] In step 410, the contour positioning module merges the accurate pixel set composed of the retained non-0 pixel connected domains to the core region ROIF to obtain a target region where the card element is located.

[0107] In the embodiments of the present application, the image to be processed is first preliminarily segmented to determine a region of interest containing the card element; then the region of interest is focused to obtain a core region of the card element; then the pixel retrieval tree is constructed based on the core region and the pixel connected domains matching the pixel retrieval tree in the expanded result region of the region of interest are retrieved by taking advantage of the similar pixel distribution pattern within the same card element; finally, the pixel connected domains with an area greater than a preset threshold are merged to the core region of the card element to obtain a target region where the card element is located, thereby improving the accuracy and positioning effect of the card element positioning. Secondly, the present application does not need data labeling and model training in the process of positioning the card element, thereby improving the efficiency of the card element positioning and reducing the cost. In addition, the method adopted by the present application has strong robustness and can still achieve good results under complex background conditions.

[0108] Optionally, in the step S204, the pixels in the core region are taken as an initial seed pixel set; the seed pixel set is iteratively updated until the seed pixel set satisfies an iteration stopping condition, and the seed pixel set obtained at the iteration stopping is taken as the target region where the card face elements are located, wherein each iteration process includes:

[0109] A candidate pixel set adjacent to the seed pixel set is adopted to construct a pixel retrieval tree; then at least one matching connected domain is obtained from the region of interest based on the pixel retrieval tree, and the at least one matching connected domain is merged into the seed pixel set.

[0110] Specifically, a candidate pixel set CS adjacent to the seed pixel set is obtained first, and color elements in the CS are selected to construct a pixel tree, which can also be constructed in combination with the candidate pixel set CS and the atomic pixel set. Then at least one matching connected domain is obtained from the region of interest based on the pixel retrieval tree, and of course, at least one matching connected domain can also be obtained from the image to be processed based on the pixel retrieval tree, which is not limited in the present application. The seed pixel set satisfying the iteration stopping condition means that the seed pixel set no longer changes.

[0111] In some embodiments, a plurality of matching pixels matching the pixel retrieval tree are retrieved from the region of interest; then at least one pixel connected domain is determined based on the obtained plurality of matching pixels; and at least one matching connected domain is selected from the at least one pixel connected domain based on the association relationship between the at least one pixel connected domain and the region of interest, respectively.

[0112] Specifically, for each pixel in the region of interest, the pixel value of the pixel is converted into a string; if the string corresponding to the pixel matches a node in the pixel retrieval tree, the pixel is a matching pixel. If the string corresponding to the pixel does not match any node in the pixel retrieval tree, the pixel is a non-matching pixel.

[0113] In actual application, for each pixel in the extended result region, the pixel value of the pixel is set as [R, G, B], and the pixel value of the pixel is converted into a string str(R)+str(G)+str(B), wherein str represents converting a number into a string and retaining 3 significant digits, for example: R=12, str(R)="012", "+" is a string concatenation operation.

[0114] It is judged whether the string str(R)+str(G)+str(B) matches the pixel retrieval tree PT, i.e., whether the node corresponding to the string str(R)+str(G)+str(B) in the pixel retrieval tree PT is 1; if yes, the pixel matches a node in the pixel retrieval tree, i.e., the pixel is a matching pixel, and the original pixel value is retained, otherwise the pixel is set to 0.

[0115] Based on the obtained plurality of matching pixels, at least one pixel connected domain can be determined. For each pixel connected domain, if the pixel connected domain has an overlapping region with the region of interest, and the ratio of the area of the overlapping region to the area of the pixel connected domain is greater than a preset threshold, the pixel connected domain is taken as a matching connected domain. The matching connected domain is merged into the seed pixel set, and the number of pixels in the seed pixel set is increased. The next iteration process is performed based on the updated seed pixel set, and the iteration stops when the seed pixel set no longer changes, and the seed pixel set obtained when the iteration stops is taken as the target region where the card face elements are located.

[0116] For example, referring to Figure 5 A flowchart of an image processing method provided by an embodiment of the present application is shown. The method is interactively executed by an image semantic segmentation module, a precise segmentation module and a pixel retrieval module in a server, and includes the following steps:

[0117] Step 501, the image semantic segmentation module reads a bank card image.

[0118] Step 502, the image semantic segmentation module performs rough semantic segmentation on the bank card image through a U-Net network to obtain a region of interest containing card face elements.

[0119] Step 503, the precise segmentation module extracts edge features E0 in the region of interest through an edge detection algorithm.

[0120] Step 504, the precise segmentation module removes edges in the edge features E0 that are connected to outside the region of interest to obtain a core region E1 of the card face elements.

[0121] Step 505, the precise segmentation module takes pixels in the core region E1 as a seed pixel set.

[0122] Step 506, the precise segmentation module takes elements adjacent to the seed pixel set as a candidate pixel set.

[0123] Step 507, the pixel retrieval module constructs a pixel retrieval tree using color elements in the candidate pixel set.

[0124] Step 508, the pixel retrieval module retrieves a set of pixel connected domains matching the pixel retrieval tree in the region of interest.

[0125] Step 509, the precise segmentation module selects a candidate connected domain from the set of pixel connected domains that has an overlapping region with the region of interest.

[0126] Step 510, the precise segmentation module determines the ratio of the area of the overlapping region to the area of the candidate connected domain.

[0127] In step 511, the accurate segmentation module determines whether the ratio is greater than a preset threshold. If yes, step 512 is executed; otherwise, step 515 is executed.

[0128] In step 512, the accurate segmentation module merges the candidate connected domain and the seed pixel set.

[0129] In step 513, the accurate segmentation module determines whether the seed pixel set changes. If yes, step 506 is executed; otherwise, step 514 is executed.

[0130] In step 514, the accurate segmentation module takes the seed pixel set as a target region where the card face element is located.

[0131] In step 515, the accurate segmentation module discards the candidate connected domain.

[0132] In the embodiment of the application, the image to be processed is first preliminarily segmented to determine a region of interest containing a card face element; then the region of interest is focused to obtain a core region of the card face element; and then, by using the feature that the internal pixel distribution patterns of the same card face element are similar, the core region is taken as an initial seed pixel set, a pixel search tree is iteratively constructed based on adjacent pixels of the seed pixel set, and a matching connected domain matching the pixel search tree in the region of interest is searched to update the seed pixel set, until the seed pixel set no longer changes, and the iteration is stopped. The seed pixel set obtained when the iteration is stopped is taken as a target region where the card face element is located, thereby improving the accuracy and positioning effect of the card face element positioning. Secondly, the application does not need to perform data labeling and model training in the process of positioning the card face element, thereby improving the efficiency of the card face element positioning and reducing the cost. In addition, the method adopted by the application has strong robustness and can still achieve good results under complex background conditions.

[0133] In any of the above embodiments, the image to be processed marked with the target region is taken as a training sample to update the deep neural network model, so as to optimize the deep neural network model in the preliminary segmentation stage, thereby improving the accuracy of the preliminary segmentation of the card face element.

[0134] Based on the same technical concept, the embodiment of the application provides a structural schematic diagram of an image processing device, as shown in Figure 6 The image processing device 600 includes:

[0135] An acquisition module 601 is configured to acquire an image to be processed.

[0136] An image segmentation module 602 is configured to preliminarily segment the image to be processed to determine a region of interest containing a card face element.

[0137] A focusing module 603 is configured to focus the region of interest to obtain a core region of the card face element.

[0138] locating module 604, configured to construct a pixel search tree based on the core region, and segment the region of interest to obtain a target region where the card face element is located by using the pixel search tree.

[0139] Optionally, the focusing module 603 is specifically configured to:

[0140] extract edge features in the region of interest;

[0141] cut out a core region of the card face element from the region of interest based on the edge features.

[0142] Optionally, the locating module 604 is specifically configured to:

[0143] take pixels in the core region as an initial seed pixel set;

[0144] iteratively update the seed pixel set until the seed pixel set meets an iteration stopping condition, and take the seed pixel set obtained when the iteration is stopped as a target region where the card face element is located, wherein each iteration process includes:

[0145] construct a pixel search tree by using a candidate pixel set adjacent to the seed pixel set;

[0146] acquire at least one matching connected domain from the region of interest based on the pixel search tree, and merge the at least one matching connected domain to the seed pixel set.

[0147] Optionally, the locating module 604 is specifically configured to:

[0148] retrieve a plurality of matching pixels from the region of interest that match the pixel search tree;

[0149] determine at least one pixel connected domain based on the plurality of obtained matching pixels;

[0150] select at least one matching connected domain from the at least one pixel connected domain based on an association relationship between the at least one pixel connected domain and the region of interest, respectively.

[0151] Optionally, the locating module 604 is specifically configured to:

[0152] convert a pixel value of each pixel in the region of interest into a string;

[0153] if the string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matching pixel.

[0154] Optionally, the locating module 604 is specifically configured to:

[0155] For each pixel connected domain, if the pixel connected domain has an overlapping region with the region of interest, and a ratio of an area of the overlapping region to an area of the pixel connected domain is greater than a preset threshold, the pixel connected domain is regarded as a matched connected domain.

[0156] Optionally, the seed pixel set satisfying the iteration stopping condition means that the seed pixel set no longer changes.

[0157] Optionally, the focusing module 603 is specifically configured to:

[0158] The length and the width of the region of interest are respectively focused on a preset value based on the unchanged center position of the region of interest, to obtain a core region of the card surface element.

[0159] Optionally, the positioning module 604 is specifically configured to:

[0160] Based on the pixels in the core region, a pixel search tree is constructed;

[0161] The region of interest is expanded to obtain an expansion result region;

[0162] Based on the pixel search tree, at least one matched connected domain is obtained from the expansion result region, and the at least one matched connected domain is merged into the core region to obtain a target region where the card surface element is located.

[0163] Optionally, the positioning module 604 is specifically configured to:

[0164] A plurality of matched pixels are searched from the expansion result region and matched with the pixel search tree;

[0165] Based on the obtained plurality of matched pixels, at least one pixel connected domain is determined;

[0166] For each pixel connected domain, if an area of the pixel connected domain is greater than or equal to a preset threshold, the pixel connected domain is regarded as a matched connected domain.

[0167] Optionally, the positioning module 604 is specifically configured to:

[0168] For each pixel in the expansion result region, a pixel value of the pixel is converted into a string;

[0169] If the string corresponding to the pixel matches one node in the pixel search tree, the pixel is a matched pixel.

[0170] Optionally, the image segmentation module 602 is specifically configured to:

[0171] A deep neural network model is used to perform preliminary segmentation on the image to be processed, and to determine the region of interest containing card elements.

[0172] Optionally, the image segmentation module 602 is further configured to:

[0173] After segmenting the region of interest using the pixel retrieval tree to obtain the target region where the card elements are located, the deep neural network model is updated using the image to be processed that marks the target region as a training sample.

[0174] In this embodiment, the image to be processed is first preliminarily segmented to determine the region of interest (ROI) containing the card elements. Then, the ROI is focused to obtain the core region of the card elements. Next, leveraging the similar pixel distribution patterns within the same card element, a pixel search tree is constructed based on the core region. This pixel search tree is then used to segment the ROI to obtain the target region where the card elements are located, thereby improving the accuracy and effectiveness of card element localization. Secondly, this application eliminates the need for data annotation and model training during card element localization, thus improving efficiency and reducing costs. Furthermore, the method employed in this application is robust and can still achieve good results under complex background conditions.

[0175] Based on the same technical concept, embodiments of this application provide a computer device, which can be... Figure 1 The terminal devices and / or servers shown, such as Figure 7 As shown, it includes at least one processor 701 and a memory 702 connected to at least one processor. In this embodiment, the specific connection medium between the processor 701 and the memory 702 is not limited. Figure 7 Taking the connection between the processor 701 and the memory 702 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0176] In this embodiment of the application, the memory 702 stores instructions that can be executed by at least one processor 701. By executing the instructions stored in the memory 702, at least one processor 701 can perform the steps of the above-described image processing method.

[0177] The processor 701 is the control center of the computer device, and can connect various parts of the computer device by using various interfaces and lines. The processor 701 can realize the card surface element positioning by running or executing instructions stored in the memory 702 and calling data stored in the memory 702. Optionally, the processor 701 can include one or more processing units. The processor 701 can integrate an application processor and a modem processor. The application processor mainly processes an operating system, a user interface, and an application program. The modem processor mainly processes wireless communication. It can be understood that the modem processor can also not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 702 can be implemented on the same chip, and in some embodiments, they can also be respectively implemented on independent chips.

[0178] The processor 701 can be a general-purpose processor, for example, a central processing unit (CPU), a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array, or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution or executed by a combination of hardware and software modules in the processor.

[0179] The memory 702, as a non-volatile computer readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 702 can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 702 is any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer device, but is not limited thereto. The memory 702 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store program instructions and / or data.

[0180] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium storing a computer program executable by a computer device, which, when executed on the computer device, causes the computer device to perform the steps of the image processing method.

[0181] Based on the same inventive concept, the embodiments of the present application provide a computer program product, which comprises a computer program stored on a computer readable storage medium, the computer program comprising program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the image processing method.

[0182] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0183] The present application is described in reference to the flowchart illustrations and / or block diagrams according to the methods, apparatus (systems) and computer program products of embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0184] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0186] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those of skill in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, the attached claims are intended to embrace all such variations and modifications as fall within the scope of the present application.

[0187] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. An image processing method, characterized by, The method comprises the following steps: acquiring a to-be-processed image; performing preliminary segmentation on the to-be-processed image to determine a region of interest containing a card face element; performing focused processing on the region of interest to obtain a core region of the card face element; taking pixels in the core region as an initial seed pixel set; iteratively updating the seed pixel set until the seed pixel set meets an iteration stopping condition, and taking the seed pixel set obtained at the time of iteration stopping as a target region where the card face element is located, wherein each iteration process comprises the following steps: constructing a pixel retrieval tree using a candidate pixel set adjacent to the seed pixel set; acquiring at least one matching connected domain from the region of interest based on the pixel retrieval tree, and merging the at least one matching connected domain to the seed pixel set.

2. The method of claim 1, wherein, The focused processing on the region of interest to obtain the core region of the card face element comprises the following steps: extracting edge features in the region of interest; based on the edge features, cutting out the core region of the card face element from the region of interest.

3. The method of claim 1, wherein, The acquiring at least one matching connected domain from the region of interest based on the pixel retrieval tree comprises the following steps: retrieving a plurality of matching pixels from the region of interest that match the pixel retrieval tree; based on the plurality of obtained matching pixels, determining at least one pixel connected domain; based on the association relationship between the at least one pixel connected domain and the region of interest respectively, selecting at least one matching connected domain from the at least one pixel connected domain.

4. The method of claim 3, wherein, The retrieving a plurality of matching pixels from the region of interest that match the pixel retrieval tree comprises the following steps: for each pixel in the region of interest, converting a pixel value of the pixel into a string; if the string corresponding to the pixel matches a node in the pixel retrieval tree, the pixel is a matching pixel.

5. The method of claim 3, wherein, The selecting at least one matching connected domain from the at least one pixel connected domain based on the association relationship between the at least one pixel connected domain and the region of interest respectively comprises the following steps: for each pixel connected domain, if the pixel connected domain and the region of interest have an overlapping region, and a ratio of an area of the overlapping region to an area of the pixel connected domain is greater than a preset threshold, the pixel connected domain is taken as a matching connected domain.

6. The method of claim 1, wherein, The seed pixel set meeting the iteration stopping condition means that the seed pixel set no longer changes.

7. The method of claim 1, wherein, The focused processing on the region of interest to obtain the core region of the card face element comprises the following steps: focusing the length and width of the region of interest on a preset value respectively without changing the center position of the region of interest, to obtain the core region of the card face element.

8. The method of claim 1, wherein, The constructing a pixel retrieval tree based on the core region comprises the following steps: based on the pixels in the core region, constructing a pixel retrieval tree; performing segmentation on the region of interest using the pixel retrieval tree to obtain a target region where the card face element is located comprises the following steps: based on the pixels in the core region, constructing a pixel retrieval tree; performing extension on the region of interest to obtain an extended result region; obtaining at least one matched connected domain from the extended result region based on the pixel search tree, and merging the at least one matched connected domain into the core region to obtain a target region where the card face element is located.

9. The method of claim 8, wherein, The obtaining at least one matched connected domain from the extended result region based on the pixel search tree comprises: retrieving a plurality of matched pixels from the extended result region that match the pixel search tree; determining at least one pixel connected domain based on the obtained plurality of matched pixels; for each pixel connected domain, if an area of the pixel connected domain is greater than or equal to a preset threshold, regarding the pixel connected domain as a matched connected domain.

10. The method of claim 9, wherein, The retrieving a plurality of matched pixels from the extended result region that match the pixel search tree comprises: for each pixel in the extended result region, converting a pixel value of the pixel into a string; if the string corresponding to the pixel matches a node in the pixel search tree, regarding the pixel as a matched pixel.

11. The method of any one of claims 1-10, wherein, The performing preliminary segmentation on the to-be-processed image to determine a region of interest containing the card face element comprises: performing preliminary segmentation on the to-be-processed image by using a deep neural network model to determine a region of interest containing the card face element.

12. The method of claim 11, wherein, After the performing segmentation on the region of interest by using the pixel search tree to obtain the target region where the card face element is located, the method further comprises: updating the deep neural network model by taking the to-be-processed image marked with the target region as a training sample.

13. An image processing apparatus characterized by comprising: comprises: an obtaining module configured to obtain a to-be-processed image; an image segmentation module configured to perform preliminary segmentation on the to-be-processed image to determine a region of interest containing a card face element; a focusing module configured to perform focusing processing on the region of interest to obtain a core region of the card face element; a positioning module configured to take pixels in the core region as an initial seed pixel set; iteratively updating the seed pixel set until the seed pixel set meets an iteration stop condition, and taking the seed pixel set obtained when the iteration stops as a target region where the card face element is located, wherein each iteration process comprises: constructing a pixel search tree by using a candidate pixel set adjacent to the seed pixel set; obtaining at least one matched connected domain from the region of interest based on the pixel search tree, and merging the at least one matched connected domain into the seed pixel set.

14. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method of any one of claims 1-12 when executing the program.

15. A computer-readable storage medium, characterized in that, The computer device stores a computer program executable by the computer device, and when the program runs on the computer device, the computer device is caused to execute the steps of the method of any one of claims 1-12.

Images