Image processing method and apparatus, device, and storage medium

Through image processing, the pixel search tree technology is used to segment the bank card images, which solves the problem of inaccurate positioning of card elements caused by manual labeling errors, and achieves efficient and accurate positioning of card elements, reducing costs and complex backgrounds error rate.

WO2024230465A1PCT designated stage expired Publication Date: 2024-11-14CHINA UNIONPAY
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/089197
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-09
Filing Date
2024-04-22
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Due to manual labeling errors and inconsistencies in bank card image processing, the deep neural network has low accuracy in semantic segmentation of bank card images, affecting the accuracy of card surface element positioning.

Method used

Using image processing method, the area of ​​interest is determined through preliminary segmentation, the core area is obtained by focusing processing, the pixel search tree is constructed based on the core area, and the area of ​​interest is segmented to obtain the target area of ​​the card surface element, so as to improve the accuracy of the card surface element positioning .

Benefits of technology

It improves the accuracy and efficiency of card element positioning, reduces the cost of data annotation and model training, and performs robustly in complex contexts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024089197_14112024_PF_FP_ABST
    Figure CN2024089197_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Embodiments of the present application relate to the technical field of artificial intelligence, and provide an image processing method and apparatus, a device, and a storage medium. The method comprises: acquiring an image to be processed (S201); carrying out preliminary segmentation on said image, and determining a region of interest containing a card surface element (S202); carrying out focusing processing on the region of interest to obtain a core region of the card surface element (S203); and constructing a pixel retrieval tree on the basis of the core region, and segmenting the region of interest by using the pixel retrieval tree, so as to obtain a target region where the card surface element is located (S204). According to the present application, the accuracy of positioning the card surface element is improved, and the positioning effect of the card surface element is improved; moreover, it is not needed to carry out data annotation and model training in the positioning process of the card surface element, so that the efficiency of positioning the card surface element is improved, and the costs are reduced. Additionally, the method used by the present application is high in robustness and still can achieve a good effect under a complex background condition.
Need to check novelty before this filing date? Find Prior Art

Description

Image processing method, device, equipment and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on May 9, 2023, with application number 202310517942.6 and invention name "An image processing method, device, equipment and storage medium", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Art

[0004] In practical applications, the size and position of bank card surface elements must meet compliance requirements, but manual judgment of such compliance is difficult. Therefore, recognition equipment is needed to locate card surface elements and automatically determine their compliance based on the positioning results and business rules.

[0005] Related technologies use deep neural networks to perform semantic segmentation on bank card images and determine the locations of card surface elements within them. However, the data used to train deep neural networks includes the original image and its corresponding semantically labeled image. Semantic labels are obtained by manually labeling each pixel in the original image. Manual classification and labeling inevitably lead to labeling errors, and inconsistencies can occur when different people label the same image. This results in low accuracy when using deep neural networks to perform semantic segmentation on bank card images, which in turn affects the accuracy of locating card surface elements.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide an image processing method, apparatus, device, and storage medium for improving the accuracy of locating elements on a card surface.

[0008] In one aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0009] Get the image to be processed;

[0010] Performing preliminary segmentation on the image to be processed to determine the region of interest containing card surface elements;

[0011] Focusing the region of interest to obtain a core region of the card element;

[0012] Based on the core area, a pixel search tree is constructed; and the region of interest is segmented using the pixel search tree to obtain a target area where the card surface element is located.

[0013] In one aspect, an embodiment of the present application provides an image processing device, comprising:

[0014] An acquisition module, used for acquiring an image to be processed;

[0015] An image segmentation module is used to perform preliminary segmentation on the image to be processed and determine the region of interest containing card surface elements;

[0016] A focusing module, configured to focus on the region of interest to obtain a core region of the card surface element;

[0017] The positioning module is used to construct a pixel search tree based on the core area; and use the pixel search tree to segment the region of interest to obtain the target area where the card surface element is located.

[0018] Optionally, the focusing module is specifically configured to:

[0019] Extracting edge features within the region of interest;

[0020] Based on the edge features, a core area of ​​the card surface element is cut out from the region of interest.

[0021] Optionally, the positioning module is specifically configured to:

[0022] Taking pixels in the core area as an initial set of seed pixels;

[0023] Iteratively updating the seed pixel set until the seed pixel set meets an iteration stop condition, and using the seed pixel set obtained when the iteration stops as the target area where the card surface element is located, wherein each iteration process includes:

[0024] Constructing a pixel search tree using a candidate pixel set adjacent to the seed pixel set;

[0025] At least one matching connected component is obtained from the region of interest based on the pixel search tree, and the at least one matching connected component is merged into the seed pixel set.

[0026] Optionally, the positioning module is specifically configured to:

[0027] Retrieving a plurality of matching pixels that match the pixel search tree from the region of interest;

[0028] Determine at least one pixel connected region based on the obtained multiple matching pixels;

[0029] Based on the association relationship between the at least one pixel connected domain and the region of interest, at least one matching connected domain is selected from the at least one pixel connected domain.

[0030] Optionally, the positioning module is specifically configured to:

[0031] For each pixel in the region of interest, converting the pixel value of the pixel into a character string;

[0032] If the character string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matching pixel.

[0033] Optionally, the positioning module is specifically configured to:

[0034] For each pixel connected domain, if there is an overlapping area between the pixel connected domain and the region of interest, and the ratio of the area of ​​the overlapping area to the area of ​​the pixel connected domain is greater than a preset threshold, the pixel connected domain is used as a matching connected domain.

[0035] Optionally, the seed pixel set meeting the iteration stop condition means that the seed pixel set no longer changes.

[0036] Optionally, the focusing module is specifically configured to:

[0037] Based on the fact that the center position of the region of interest does not change, the length and width of the region of interest are focused on preset values ​​respectively to obtain the core area of ​​the card surface element.

[0038] Optionally, the positioning module is specifically configured to:

[0039] Constructing a pixel search tree based on the pixels within the core area;

[0040] Expanding the region of interest to obtain an expanded result region;

[0041] At least one matching connected domain is obtained from the expanded result area based on the pixel search tree, and the at least one matching connected domain is merged into the core area to obtain the target area where the card surface element is located.

[0042] Optionally, the positioning module is specifically configured to:

[0043] Retrieving a plurality of matching pixels that match the pixel search tree from the expanded result area;

[0044] Determine at least one pixel connected region based on the obtained multiple matching pixels;

[0045] For each pixel connected domain, if the area of ​​the pixel connected domain is greater than or equal to a preset threshold, the pixel connected domain is used as a matching connected domain.

[0046] Optionally, the positioning module is specifically configured to:

[0047] For each pixel in the extended result area, converting the pixel value of the pixel into a character string;

[0048] If the character string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matching pixel.

[0049] Optionally, the image segmentation module is specifically used to:

[0050] A deep neural network model is used to perform preliminary segmentation on the image to be processed to determine the region of interest containing the card surface elements.

[0051] Optionally, the image segmentation module is further used to:

[0052] The pixel search tree is used to segment the region of interest to obtain the target area where the card surface element is located, and then the image to be processed that marks the target area is used as a training sample to update the deep neural network model.

[0053] On the one hand, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned image processing method when executing the program.

[0054] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the program is run on the computer device, the computer device executes the steps of the above-mentioned image processing method.

[0055] On the one hand, an embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer performs the steps of the above-mentioned image processing method.

[0056] In an embodiment of the present application, a preliminary segmentation of the image to be processed is first performed to determine the region of interest containing the card surface elements; then, the region of interest is focused to obtain the core region of the card surface elements; then, utilizing the similarity of pixel distribution patterns within the same card surface element, a pixel search tree is constructed based on the core region; and the pixel search tree is used to segment the region of interest to obtain the target region where the card surface elements are located, thereby improving the accuracy of locating the card surface elements and improving the locating effect of the card surface elements. Secondly, the present application does not require data annotation and model training in the process of locating the card surface elements, thereby improving the efficiency of locating the card surface elements and reducing costs. In addition, the method adopted in the present application is highly robust and can still achieve good results under complex background conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] FIG1 is a schematic diagram of a system architecture provided in an embodiment of the present application;

[0059] FIG2 is a flowchart of an image processing method according to an embodiment of the present application;

[0060] FIG3 is a schematic diagram of the structure of a deep neural network model provided in an embodiment of the present application;

[0061] FIG4 is a second flow chart of an image processing method provided in an embodiment of the present application;

[0062] FIG5 is a third flow chart of an image processing method provided in an embodiment of the present application;

[0063] FIG6 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0064] FIG7 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and beneficial effects of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0066] For ease of understanding, the terms involved in the embodiments of the present invention are explained below.

[0067] ROI: Region of Interest, region of interest.

[0068] Faster R-CNN: Faster Region-CNN, Fast Regional Convolutional Neural Network, a fast end-to-end object detection deep neural network.

[0069] YOLO: You Look once, a commonly used algorithm in target detection algorithms.

[0070] U-net: u-shaped network, U-shaped convolutional neural network.

[0071] FCN: Fully Convolutional Networks, full convolutional neural network.

[0072] Refer to Figure 1, which is a system architecture diagram applicable to an embodiment of the present application. The system architecture includes at least a terminal device 101 and a server 102. The number of terminal devices 101 can be one or more, and the number of servers 102 can also be one or more. This application does not specifically limit the number of terminal devices 101 and servers 102.

[0073] An application for locating card elements is pre-installed in the terminal device 101, where the application is a client application, a web application, a mini-program application, etc. The terminal device 101 can be a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart home appliance, an intelligent voice interaction device, an intelligent vehicle-mounted device, etc., but is not limited thereto.

[0074] Server 102 is the background server of the application. Server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal device 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0075] The image processing method in the embodiment of the present application can be executed by the terminal device 101, can be executed by the server 102, or can be executed interactively by the terminal device 101 and the server 102.

[0076] Based on the system architecture diagram shown in FIG1 , an embodiment of the present application provides a process of an image processing method, as shown in FIG2 . The process of the method is executed by a computer device, which may be the terminal device 101 and / or the server 102 shown in FIG1 , and includes the following steps:

[0077] Step S201: Obtain an image to be processed.

[0078] Specifically, the image to be processed may be an image of a bank card, a shopping card, a membership card, etc. The image to be processed may be obtained by taking a photo or may be a design drawing, and this application does not make any specific limitation on this.

[0079] Step S202: performing preliminary segmentation on the image to be processed to determine the region of interest containing the card surface elements.

[0080] In some embodiments, a deep neural network model is used to perform preliminary segmentation on the image to be processed to determine the region of interest containing the card surface elements, wherein the deep neural network model can be a deep network model such as U-Net, FCN, Faster R-CNN, yolo, etc.

[0081] In some embodiments, in addition to using a deep neural network model to obtain a region of interest containing card surface elements, the type of card surface elements can also be obtained.

[0082] Taking the deep neural network model Faster R-CNN as an example, see Figure 3, the deep neural network model 300 includes a convolution layer 301, a candidate region generation network (RPN) 302, a target area pooling layer 303, and a classifier 304. The bank card image is input into the convolution layer 301, the convolution layer 301 extracts the image features of the bank card image, and then shares the image features with the candidate region generation network 302 and the target area pooling layer 303. The candidate region generation network 302 generates regional candidate image blocks based on the image features to realize the positioning of the region of interest of the card surface element. The target area pooling layer 303 extracts the feature map of the region of interest based on the image features. The classifier 304 outputs the type of the card surface element and the region of interest based on the regional candidate image blocks and the feature map of the region of interest.

[0083] Step S203: focusing the region of interest to obtain the core region of the card surface element.

[0084] Specifically, focusing on the region of interest refers to: cropping the region of interest to remove the background area in the region of interest, or removing the background area in the region of interest and the peripheral area of ​​the card surface element, so as to obtain the core area of ​​the card surface element.

[0085] In some embodiments, based on the unchanged center position of the region of interest, the length and width of the region of interest are focused on preset values ​​to obtain the core area of ​​the card surface element.

[0086] Specifically, when the card surface element contains fewer color features, such as the chip logo on a bank card, the length and width of the area of ​​interest are focused to preset values ​​to obtain the core area of ​​the card surface element.

[0087] For example, a bank card image is input into a deep neural network model. The model outputs the region of interest (ROI) of the card surface element as follows: ROI = [(xroimin, yroimin), (xroimax, yroimax)]. Based on the unchanged center position of the ROI, the length and width of the ROI are focused by a preset value a, and the core region of the card surface element is obtained as follows: ROIF = [(xroimin+a, yroimin+a), (xroimax-a, yroimax-a)]. The core region ROIF contains all pixels of the card surface element.

[0088] In some embodiments, edge features within the region of interest are extracted; and based on the edge features, a core area of ​​the card surface element is cut out from the region of interest.

[0089] Specifically, when the card surface element contains many color features, such as the bank logo on a bank card, the edge feature E0 in the region of interest is extracted, and then the edges in the edge feature E0 that are connected to the outside of the region of interest are removed to obtain the core area E1 of the card surface element.

[0090] Step S204: constructing a pixel search tree based on the core area; and using the pixel search tree, segmenting the region of interest to obtain the target area where the card surface elements are located.

[0091] Specifically, the pixel search tree includes multiple nodes, each corresponding to a string. Using the pixel search tree, the region of interest is finely segmented to obtain the precise area where the card surface element is located.

[0092] In an embodiment of the present application, a preliminary segmentation of the image to be processed is first performed to determine the region of interest containing the card surface elements; then, the region of interest is focused to obtain the core region of the card surface elements; then, utilizing the similarity of pixel distribution patterns within the same card surface element, a pixel search tree is constructed based on the core region; and the pixel search tree is used to segment the region of interest to obtain the target region where the card surface elements are located, thereby improving the accuracy of locating the card surface elements and improving the locating effect of the card surface elements. Secondly, the present application does not require data annotation and model training in the process of locating the card surface elements, thereby improving the efficiency of locating the card surface elements and reducing costs. In addition, the method adopted in the present application is highly robust and can still achieve good results under complex background conditions.

[0093] Optionally, in the above step S204, a pixel retrieval tree is constructed based on the pixels in the core area; then the area of ​​interest is expanded to obtain an extended result area; at least one matching connected domain is obtained from the extended result area based on the pixel retrieval tree, and at least one matching connected domain is merged into the core area to obtain the target area where the card surface elements are located.

[0094] Specifically, the region is expanded with the region of interest ROI as the center, and the obtained expanded result area is: ROIE = [(xroimin-b, yroimin-b), (xroimax+b, yroimax+b)], so that the card surface elements are completely contained in the ROIE area.

[0095] Multiple matching pixels that match the pixel search tree are retrieved from the expanded result area. Then, based on the obtained multiple matching pixels, at least one pixel connected domain is determined. For each pixel connected domain, if the area of ​​the pixel connected domain is greater than or equal to a preset threshold, the pixel connected domain is considered as a matching connected domain.

[0096] Specifically, for each pixel in the expanded result area, the pixel value of the pixel is converted into a character string; if the character string corresponding to the pixel matches a node in the pixel retrieval tree, the pixel is a matching pixel; if the character string corresponding to the pixel does not match any node in the pixel retrieval tree, the pixel is a non-matching pixel.

[0097] In practical applications, for each pixel in the extended result area, the pixel value of the pixel is set to [R, G, B], and the pixel value of the pixel is converted into a string str(R)+str(G)+str(B), where str means converting a number into a string and retaining 3 valid digits. For example: R=12, then str(R)="012", and "+" is a string concatenation operation.

[0098] Determine whether the string str(R)+str(G)+str(B) matches the pixel retrieval tree PT, that is, whether the node corresponding to the string str(R)+str(G)+str(B) in the pixel retrieval tree PT is 1; if so, the pixel matches a node in the pixel retrieval tree, that is, the pixel is a matching pixel and the original pixel value is retained, otherwise the pixel is set to 0.

[0099] Based on the multiple matching pixels obtained, at least one connected domain of pixels is determined, and the area of ​​each connected domain of pixels is calculated. When the area of ​​a connected domain of pixels is less than a preset threshold smin, the pixels within the connected domain of pixels are set to 0. Finally, the precise pixel set consisting of the retained connected domain of non-zero pixels is merged into the core area to obtain the target area where the card surface elements are located. The coordinate range of the pixels in the precise pixel set [(xmin, ymin), (xmax, ymax)] is used as the location of the target area in the bank card image.

[0100] For example, referring to FIG4 , which is a flowchart of an image processing method provided in an embodiment of the present application, the method is interactively executed by a card surface element positioning module, an element segmentation module, a pixel retrieval module, and a contour positioning module in a server, and includes the following steps:

[0101] Step 401: The card surface element positioning module reads the bank card image.

[0102] In step 402, the card surface element positioning module locates the regions of interest and types of card surface elements in the bank card image through Faster R-CNN.

[0103] Step 403: The element segmentation module automatically focuses on the region of interest as a boundary to obtain the core region ROIF of the card surface element.

[0104] Specifically, when the type of the card surface element is a preset element type with fewer color features (such as a chip logo), step 403 and subsequent steps are executed; otherwise, the subsequent steps are not executed.

[0105] Step 404: The pixel search module constructs a pixel search tree based on the pixels in the core area ROIF.

[0106] Step 405: The element segmentation module performs region expansion with the region of interest as the center to obtain an expanded result region ROIE.

[0107] Step 406: The pixel search module searches for all pixels in the expanded result region ROIE in the pixel search tree to obtain matching pixels in the expanded result region ROIE.

[0108] In step 407 , the element segmentation module retains the pixel values ​​of the matching pixels and sets the pixel values ​​of the non-matching pixels to 0.

[0109] In step 408 , the element segmentation module determines at least one pixel connected component based on the obtained multiple matching pixels.

[0110] In step 409 , the element segmentation module sets the pixel value of each pixel in the connected region of pixels whose area is smaller than a preset threshold smin to 0.

[0111] In step 410 , the outline positioning module merges the retained accurate pixel set consisting of the connected domain of non-zero pixels into the core region ROIF to obtain the target region where the card surface elements are located.

[0112] In an embodiment of the present application, a preliminary segmentation is first performed on the image to be processed to determine the region of interest containing the card surface elements; then the region of interest is focused to obtain the core region of the card surface elements; then, by utilizing the similarity of the pixel distribution patterns within the same card surface element, a pixel search tree is constructed based on the core region, and the pixel connected domain that matches the pixel search tree in the extended result area of ​​the region of interest is retrieved, and finally, the pixel connected domain with an area greater than a preset threshold is merged into the core region of the card surface element to obtain the target region where the card surface element is located, thereby improving the accuracy and positioning effect of the card surface element positioning. Secondly, the present application does not require data annotation and model training in the process of locating the card surface elements, thereby improving the efficiency of card surface element positioning and reducing costs. In addition, the method adopted in the present application is highly robust and can still achieve good results under complex background conditions.

[0113] Optionally, in step S204, the pixels in the core area are used as an initial seed pixel set; the seed pixel set is iteratively updated until the seed pixel set meets the iteration stop condition, and the seed pixel set obtained when the iteration stops is used as the target area where the card surface element is located, wherein each iteration process includes:

[0114] A pixel search tree is constructed using a candidate pixel set adjacent to a seed pixel set; then, at least one matching connected domain is obtained from the region of interest based on the pixel search tree, and the at least one matching connected domain is merged into the seed pixel set.

[0115] Specifically, first obtain a candidate pixel set CS adjacent to the seed pixel set, select the color elements in CS to construct a pixel tree, or combine the candidate pixel set CS and the atomic pixel set to construct a pixel tree. Then, based on the pixel retrieval tree, obtain at least one matching connected domain from the region of interest. Of course, it is also possible to obtain at least one matching connected domain from the image to be processed based on the pixel retrieval tree. This is not specifically limited in this application. The seed pixel set satisfies the iteration stop condition when the seed pixel set no longer changes.

[0116] In some embodiments, multiple matching pixels that match the pixel retrieval tree are retrieved from the region of interest; then, based on the obtained multiple matching pixels, at least one pixel connected domain is determined; and then, based on the association relationship between the at least one pixel connected domain and the region of interest, at least one matching connected domain is selected from the at least one pixel connected domain.

[0117] Specifically, for each pixel in the region of interest, the pixel value is converted into a string; if the string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matched pixel. If the string corresponding to the pixel does not match any node in the pixel search tree, the pixel is a non-matched pixel.

[0118] In practical applications, for each pixel in the extended result area, the pixel value of the pixel is set to [R, G, B], and the pixel value of the pixel is converted into a string str(R)+str(G)+str(B), where str means converting a number into a string and retaining 3 valid digits. For example: R=12, then str(R)="012", and "+" is a string concatenation operation.

[0119] Determine whether the string str(R)+str(G)+str(B) matches the pixel retrieval tree PT, that is, whether the node corresponding to the string str(R)+str(G)+str(B) in the pixel retrieval tree PT is 1; if so, the pixel matches a node in the pixel retrieval tree, that is, the pixel is a matching pixel and the original pixel value is retained, otherwise the pixel is set to 0.

[0120] Based on the obtained multiple matching pixels, at least one connected domain of pixels can be determined. For each connected domain of pixels, if there is an overlap between the connected domain of pixels and the region of interest, and the ratio of the area of ​​the overlapping area to the area of ​​the connected domain of pixels is greater than a preset threshold, the connected domain of pixels is considered a matching connected domain. The matching connected domain is then merged into a seed pixel set, which now contains more pixels. The next iteration is then performed based on the updated seed pixel set. The iteration stops when the seed pixel set no longer changes, and the seed pixel set obtained at the end of the iteration is used as the target area for the card surface element.

[0121] For example, referring to FIG5 , which is a flowchart of an image processing method provided in an embodiment of the present application, the method is interactively executed by an image semantic segmentation module, a precise segmentation module, and a pixel retrieval module in a server, and includes the following steps:

[0122] Step 501: The image semantic segmentation module reads the bank card image.

[0123] In step 502, the image semantic segmentation module performs rough semantic segmentation on the bank card image through the U-Net network to obtain the region of interest containing the card surface elements.

[0124] Step 503: The precise segmentation module extracts edge features E0 in the region of interest using an edge detection algorithm.

[0125] In step 504 , the precise segmentation module removes the edges in the edge feature E0 that are connected to the outside of the region of interest, and obtains the core region E1 of the card surface element.

[0126] Step 505 : The precise segmentation module uses the pixels in the core area E1 as a set of seed pixels.

[0127] In step 506 , the precise segmentation module takes the elements adjacent to the seed pixel set as a candidate pixel set.

[0128] Step 507: The pixel search module constructs a pixel search tree using the color elements in the candidate pixel set.

[0129] Step 508: The pixel search module searches for a set of pixel connected components that matches the pixel search tree in the region of interest.

[0130] In step 509 , the precise segmentation module selects a candidate connected domain that overlaps with the region of interest from the pixel connected domain set.

[0131] In step 510 , the precise segmentation module determines the ratio of the area of ​​the overlapping region to the area of ​​the candidate connected component.

[0132] In step 511 , the precise segmentation module determines whether the ratio is greater than a preset threshold. If so, step 512 is executed; otherwise, step 515 is executed.

[0133] In step 512 , the precise segmentation module merges the candidate connected component with the seed pixel set.

[0134] In step 513 , the precise segmentation module determines whether the seed pixel set has changed. If so, step 506 is executed; otherwise, step 514 is executed.

[0135] In step 514, the precise segmentation module uses the seed pixel set as the target area where the card surface elements are located.

[0136] Step 515: The precise segmentation module discards the candidate connected components.

[0137] In an embodiment of the present application, a preliminary segmentation is first performed on the image to be processed to determine the region of interest containing the card surface elements; then, the region of interest is focused to obtain the core region of the card surface elements; then, utilizing the similarity of the pixel distribution patterns within the same card surface element, the core region is used as the initial seed pixel set, a pixel search tree is iteratively constructed based on the adjacent pixels of the seed pixel set, and the matching connected domain that matches the pixel search tree in the region of interest is retrieved to update the seed pixel set, and the iteration is stopped until the seed pixel set no longer changes. The seed pixel set obtained when the iteration is stopped is used as the target region where the card surface elements are located, thereby improving the accuracy and positioning effect of the card surface element positioning. Secondly, the present application does not require data annotation and model training in the process of locating the card surface elements, thereby improving the efficiency of card surface element positioning and reducing costs. In addition, the method adopted in the present application is highly robust and can still achieve good results under complex background conditions.

[0138] In any of the above embodiments, the image to be processed with the marked target area is used as a training sample to update the deep neural network model to optimize the deep neural network model in the preliminary segmentation stage, thereby improving the accuracy of the preliminary segmentation of card surface elements.

[0139] Based on the same technical concept, an embodiment of the present application provides a structural diagram of an image processing device, as shown in FIG6 . The image processing device 600 includes:

[0140] An acquisition module 601 is used to acquire an image to be processed;

[0141] An image segmentation module 602 is configured to perform preliminary segmentation on the image to be processed and determine a region of interest containing card surface elements;

[0142] A focusing module 603 is configured to focus on the region of interest to obtain a core region of the card element;

[0143] The positioning module 604 is configured to construct a pixel search tree based on the core area; and segment the region of interest using the pixel search tree to obtain a target region where the card surface element is located.

[0144] Optionally, the focusing module 603 is specifically configured to:

[0145] Extracting edge features within the region of interest;

[0146] Based on the edge features, a core area of ​​the card surface element is cut out from the region of interest.

[0147] Optionally, the positioning module 604 is specifically configured to:

[0148] Taking pixels in the core area as an initial set of seed pixels;

[0149] Iteratively updating the seed pixel set until the seed pixel set meets an iteration stop condition, and using the seed pixel set obtained when the iteration stops as the target area where the card surface element is located, wherein each iteration process includes:

[0150] Constructing a pixel search tree using a candidate pixel set adjacent to the seed pixel set;

[0151] At least one matching connected component is obtained from the region of interest based on the pixel search tree, and the at least one matching connected component is merged into the seed pixel set.

[0152] Optionally, the positioning module 604 is specifically configured to:

[0153] Retrieving a plurality of matching pixels that match the pixel search tree from the region of interest;

[0154] Determine at least one pixel connected region based on the obtained multiple matching pixels;

[0155] Based on the association relationship between the at least one pixel connected domain and the region of interest, at least one matching connected domain is selected from the at least one pixel connected domain.

[0156] Optionally, the positioning module 604 is specifically configured to:

[0157] For each pixel in the region of interest, converting the pixel value of the pixel into a character string;

[0158] If the character string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matching pixel.

[0159] Optionally, the positioning module 604 is specifically configured to:

[0160] For each pixel connected domain, if there is an overlapping area between the pixel connected domain and the region of interest, and the ratio of the area of ​​the overlapping area to the area of ​​the pixel connected domain is greater than a preset threshold, the pixel connected domain is used as a matching connected domain.

[0161] Optionally, the seed pixel set meeting the iteration stop condition means that the seed pixel set no longer changes.

[0162] Optionally, the focusing module 603 is specifically configured to:

[0163] Based on the fact that the center position of the region of interest does not change, the length and width of the region of interest are focused on preset values ​​respectively to obtain the core area of ​​the card surface element.

[0164] Optionally, the positioning module 604 is specifically configured to:

[0165] Constructing a pixel search tree based on the pixels within the core area;

[0166] Expanding the region of interest to obtain an expanded result region;

[0167] At least one matching connected domain is obtained from the expanded result area based on the pixel search tree, and the at least one matching connected domain is merged into the core area to obtain the target area where the card surface element is located.

[0168] Optionally, the positioning module 604 is specifically configured to:

[0169] Retrieving a plurality of matching pixels that match the pixel search tree from the expanded result area;

[0170] Determine at least one pixel connected region based on the obtained multiple matching pixels;

[0171] For each pixel connected domain, if the area of ​​the pixel connected domain is greater than or equal to a preset threshold, the pixel connected domain is used as a matching connected domain.

[0172] Optionally, the positioning module 604 is specifically configured to:

[0173] For each pixel in the extended result area, converting the pixel value of the pixel into a character string;

[0174] If the character string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matching pixel.

[0175] Optionally, the image segmentation module 602 is specifically configured to:

[0176] A deep neural network model is used to perform preliminary segmentation on the image to be processed to determine the region of interest containing the card surface elements.

[0177] Optionally, the image segmentation module 602 is further configured to:

[0178] The pixel search tree is used to segment the region of interest to obtain the target area where the card surface element is located, and then the image to be processed that marks the target area is used as a training sample to update the deep neural network model.

[0179] In an embodiment of the present application, a preliminary segmentation of the image to be processed is first performed to determine the region of interest containing the card surface elements; then, the region of interest is focused to obtain the core region of the card surface elements; then, utilizing the similarity of pixel distribution patterns within the same card surface element, a pixel search tree is constructed based on the core region; and the pixel search tree is used to segment the region of interest to obtain the target region where the card surface elements are located, thereby improving the accuracy of locating the card surface elements and improving the locating effect of the card surface elements. Secondly, the present application does not require data annotation and model training in the process of locating the card surface elements, thereby improving the efficiency of locating the card surface elements and reducing costs. In addition, the method adopted in the present application is highly robust and can still achieve good results under complex background conditions.

[0180] Based on the same technical concept, an embodiment of the present application provides a computer device, which may be the terminal device and / or server shown in FIG1 . As shown in FIG7 , the computer device includes at least one processor 701 and a memory 702 connected to the at least one processor. The embodiment of the present application does not limit the specific connection medium between the processor 701 and the memory 702. FIG7 takes the connection between the processor 701 and the memory 702 via a bus as an example. Buses can be divided into address buses, data buses, control buses, etc.

[0181] In the embodiment of the present application, the memory 702 stores instructions that can be executed by at least one processor 701. The at least one processor 701 can perform the steps of the above-mentioned image processing method by executing the instructions stored in the memory 702.

[0182] The processor 701 is the control center of the computer device. It can connect the various components of the computer device using various interfaces and lines. It can achieve card surface element positioning by running or executing instructions stored in the memory 702 and calling data stored in the memory 702. Optionally, the processor 701 may include one or more processing units. The processor 701 may integrate an application processor and a modem processor. The application processor mainly processes the operating system, user interface, and application programs, while the modem processor mainly handles wireless communications. It is understood that the modem processor may not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 702 may be implemented on the same chip. In some embodiments, they may also be implemented on separate chips.

[0183] The processor 701 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0184] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 702 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 702 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer device, but is not limited thereto. The memory 702 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0185] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned image processing method.

[0186] Based on the same inventive concept, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device performs the steps of the above-mentioned image processing method.

[0187] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0188] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer device or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0189] These computer program instructions may also be stored in a computer-readable memory that can direct a computer device or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0190] These computer program instructions can also be loaded onto a computer device or other programmable data processing device, so that a series of operating steps are executed on the computer device or other programmable device to produce a process implemented by the computer device, so that the instructions executed on the computer device or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0191] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0192] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. An image processing method, characterized in that: include: Get the image to be processed; Performing preliminary segmentation on the image to be processed to determine the region of interest containing card surface elements; Focusing the region of interest to obtain a core region of the card surface element; Based on the core area, construct a pixel retrieval tree; Furthermore, the pixel search tree is used to segment the region of interest to obtain the target region where the card surface element is located.

2. The method according to claim 1, characterized in that The focusing process on the region of interest to obtain the core region of the card surface element includes: Extract edge features within the region of interest; Based on the edge features, a core area of ​​the card surface element is cut out from the region of interest.

3. The method according to claim 1, characterized in that Based on the core area, constructing a pixel search tree; And, using the pixel search tree to segment the region of interest to obtain the target region where the card surface element is located, including: Using pixels in the core area as an initial set of seed pixels; Iteratively update the seed pixel set until the seed pixel set meets the iteration stop condition, and use the seed pixel set obtained when the iteration stops as the target area where the card surface element is located, wherein each iteration process includes: Constructing a pixel search tree using a candidate pixel set adjacent to the seed pixel set; At least one matching connected domain is obtained from the region of interest based on the pixel search tree, and the at least one matching connected domain is merged into the seed pixel set.

4. The method according to claim 3, characterized in that The acquiring at least one matching connected domain from the region of interest based on the pixel search tree comprises: Retrieving a plurality of matching pixels matching the pixel retrieval tree from the region of interest; Based on the obtained multiple matching pixels, determining at least one pixel connected domain; Based on the association relationship between the at least one pixel connected domain and the region of interest, at least one matching connected domain is selected from the at least one pixel connected domain.

5. The method according to claim 4, characterized in that The step of retrieving a plurality of matching pixels matching the pixel search tree from the region of interest comprises: For each pixel in the region of interest, converting the pixel value of the pixel into a character string; If the character string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matching pixel.

6. The method according to claim 4, characterized in that The selecting at least one matching connected domain from the at least one pixel connected domain based on the association relationship between the at least one pixel connected domain and the region of interest respectively comprises: For each pixel connected domain, if there is an overlapping area between the pixel connected domain and the region of interest, and the ratio of the area of ​​the overlapping area to the area of ​​the pixel connected domain is greater than a preset threshold, the pixel connected domain is used as a matching connected domain.

7. The method according to claim 3, characterized in that The seed pixel set satisfies the iteration stop condition when: the seed pixel set does not change any more.

8. The method according to claim 1, characterized in that The focusing process on the region of interest to obtain the core region of the card surface element includes: Based on the unchanged center position of the region of interest, the length and width of the region of interest are focused on preset values ​​respectively to obtain the core area of ​​the card surface element.

9. The method according to claim 1, characterized in that Based on the core area, constructing a pixel search tree; And, using the pixel search tree to segment the region of interest to obtain the target region where the card surface element is located, including: Based on the pixels in the core area, construct a pixel search tree; Expanding the region of interest to obtain an expanded result region; At least one matching connected domain is obtained from the extended result area based on the pixel search tree, and the at least one matching connected domain is merged into the core area to obtain the target area where the card surface element is located.

10. The method according to claim 9, characterized in that The acquiring at least one matching connected domain from the extended result area based on the pixel search tree comprises: Retrieving a plurality of matching pixels matching the pixel search tree from the extended result area; Based on the obtained multiple matching pixels, determining at least one pixel connected domain; For each pixel connected domain, if the area of ​​the pixel connected domain is greater than or equal to a preset threshold, the pixel connected domain is used as a matching connected domain.

11. The method according to claim 10, characterized in that The step of retrieving a plurality of matching pixels matching the pixel search tree from the extended result area comprises: For each pixel in the extended result area, converting the pixel value of the pixel into a character string; If the character string corresponding to the pixel matches a node in the pixel search tree, the pixel is a matching pixel.

12. The method according to any one of claims 1 to 11, characterized in that: The step of performing preliminary segmentation on the image to be processed to determine the region of interest containing the card surface elements includes: A deep neural network model is used to perform preliminary segmentation on the image to be processed to determine the region of interest containing the card surface elements.

13. The method according to claim 12, characterized in that After the pixel search tree is used to segment the region of interest to obtain the target region where the card surface element is located, the method further includes: The deep neural network model is updated by taking the image to be processed with the target area marked as a training sample.

14. An image processing device, characterized in that: include: An acquisition module, used for acquiring an image to be processed; An image segmentation module is used to perform preliminary segmentation on the image to be processed and determine the region of interest containing the card surface elements; A focusing module, used for focusing the region of interest to obtain a core region of the card surface element; A positioning module, used for constructing a pixel retrieval tree based on the core area; Furthermore, the pixel search tree is used to segment the region of interest to obtain the target region where the card surface element is located.

15. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of any one of the methods of claims 1 to 13 are implemented.

16. A computer-readable storage medium, characterized in that: It stores a computer program executable by a computer device. When the program is run on the computer device, the computer device executes the steps of any method described in claims 1 to 13.

17. A computer program product, characterized in that The computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to execute the steps of the method according to any one of claims 1 to 13.

Citation Information

Patent Citations

  • Method and device for recognizing object in image

    CN103324949A

  • License plate segmentation method and device, computer equipment and computer readable storage medium

    CN110232381A

  • Image processing method and device based on artificial intelligence and computer equipment

    CN112330688A

  • Card surface element identifying and positioning method and device

    CN115797391A

  • Image processing method and device, equipment and storage medium

    CN116612276A