Card ticket image segmentation method, device, equipment and program product

Abstract

The application discloses a card ticket image segmentation method, device, equipment and program product, and is used for solving the problems of inaccurate edge positioning, poor adaptability and error accumulation in a complex scene. The scheme comprises the following steps: determining an edge starting point of a target card ticket image. Starting from the edge starting point, performing multiple rounds of edge tracking on the target card ticket image to determine an edge path; wherein each round of edge tracking comprises the following steps: taking a current edge point as a center, intercepting a local image in the target card ticket image, and generating a current state vector based on the local image; inputting the current state vector into a first intelligent agent to predict multiple candidate positions of a next edge point; inputting the current state vector into a second intelligent agent to predict an offset direction from the current edge point to the next edge point; and determining the next edge point based on the offset direction and the multiple candidate positions determined in the current round. A segmentation mask is constructed based on the determined edge path, and the target card ticket image is segmented based on the segmentation mask.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, device and program product for segmenting card and ticket images. Background Technology

[0002] Image segmentation is one of the core tasks in computer vision, aiming to divide an image into regions or object contours with specific meaning, thereby extracting the target region of interest. In the automatic processing of cards and documents (such as ID cards, driver's licenses, etc.), accurate image segmentation can effectively separate the document subject from the complex background, providing clear and standardized input for subsequent OCR recognition, information extraction, and quality inspection, and is a key prerequisite for improving the overall system automation and recognition accuracy.

[0003] However, in practical applications, card and document images often suffer from various interferences due to complex shooting environments, such as mixed background textures, uneven lighting, partial occlusion, and tilted or curved documents. Existing image segmentation techniques still face significant challenges in dealing with such complex scenarios: traditional methods are susceptible to noise interference and have limited edge localization capabilities; while learning-based methods often suffer from insufficient adaptability, loss of detail, or error accumulation under complex textures and multi-scale structures, resulting in inaccurate segmentation boundaries and poor integrity, directly affecting the reliability of subsequent processing. Summary of the Invention

[0004] This application proposes a method, apparatus, device, and program product for segmenting card and document images, aiming to solve the problems of insufficient segmentation accuracy and robustness in complex scenarios caused by inaccurate edge positioning, poor adaptability, and error accumulation in existing technologies. Accordingly, the technical solution of this application is as follows: Firstly, a method for segmenting card / ticket images is provided, including: Determine the starting point of the edge of the target card / document image; Starting from the edge origin, multiple rounds of edge tracking are performed on the target card / document image to determine the edge path. Each round of edge tracking includes: capturing a local image of the target card / document image centered on the current edge point, and generating a current state vector based on the local image; inputting the current state vector into a first agent to predict multiple candidate positions of the next edge point; inputting the current state vector into a second agent to predict the offset direction from the current edge point to the next edge point; and determining the next edge point based on the offset direction determined in this round and the multiple candidate positions. A segmentation mask is constructed based on the determined edge path, and the segmentation mask is used to segment the target card / document image.

[0005] Secondly, a card / ticket image segmentation device is provided, comprising: The starting point determination module determines the edge starting point of the target card / document image; The edge tracking module performs multiple rounds of edge tracking on the target card / document image, starting from the edge origin, to determine the edge path. Each round of edge tracking includes: capturing a local image of the target card / document image centered on the current edge point, and generating a current state vector based on the local image; inputting the current state vector into a first agent to predict multiple candidate positions for the next edge point; inputting the current state vector into a second agent to predict the offset direction from the current edge point to the next edge point; and determining the next edge point based on the offset direction determined in this round and the multiple candidate positions. The image segmentation module constructs a segmentation mask based on the determined edge paths and uses the segmentation mask to segment the target card / document image.

[0006] Thirdly, embodiments of this application provide an electronic device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method described in the first aspect.

[0007] Fourthly, a computer program product is provided, the computer program product including a computer-readable storage medium storing a computer program operable to cause a computer to perform the method described in the first aspect.

[0008] This embodiment first determines the starting point of the edge of the target card / document image, and then performs multiple rounds of edge tracing from this starting point to gradually construct a complete edge path. In each round of tracing, the system captures a local image centered on the current edge point and extracts features to construct a state vector. This state vector is simultaneously input into two parallel agents for collaborative decision-making: the first agent predicts multiple candidate coordinates for the next position, focusing on the probability distribution of spatial location; the second agent focuses on predicting the offset direction from the current point to the next point to capture the local geometric characteristics of the edge direction. By fusing information from both the candidate position and the offset direction, the system can more accurately determine the location of the next edge point. This dual-agent collaborative and information fusion mechanism effectively alleviates three key problems in existing technologies in principle: First, the joint analysis of position candidates and orientation constraints enables mutual verification and optimization of position decisions at each step, significantly improving edge localization accuracy. Second, the state vector dynamically constructed based on local images allows the decision-making process to adaptively focus on the local features of the current edge, thereby enhancing the system's robustness in complex backgrounds or scenarios with changing target poses. Third, thanks to the detailed modeling of the local image context at each step and the fusion of two-dimensional information, the probability of misjudging a single edge point is greatly reduced, effectively preventing the propagation and accumulation of errors in subsequent tracking processes, and ensuring the overall consistency and segmentation accuracy of long-path edge reconstruction. Finally, based on the constructed high-precision edge path, the system can generate a segmentation mask that accurately selects the target region, achieving efficient and stable segmentation of the target region in card and ticket images, significantly improving its segmentation performance in real and complex scenarios. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating the card and ticket image segmentation method according to an embodiment of this application.

[0011] Figure 2 This is a schematic diagram of the structure of the card and document image segmentation device according to an embodiment of this application.

[0012] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0014] One embodiment of this application provides a method for segmenting card and ticket images. Figure 1 This is a flowchart illustrating the image segmentation method for this card / document, including: S101, Determine the starting point of the edge of the target card document image.

[0015] In this embodiment, the target document image refers to the original image of the document to be segmented (such as an ID card, driver's license, etc.) collected in a real-world application scenario. These images often have various interferences due to complex shooting conditions, such as mixed background textures, uneven lighting, partial occlusion, and tilted or curved documents, making direct segmentation difficult. Therefore, before starting edge tracking, it is necessary to determine a reliable starting point for the edge to gradually construct a closed and accurate document outline.

[0016] The significance of determining the starting point of the edge lies in providing a stable initial position and a benchmark for subsequent multi-round edge tracking. An inappropriate starting point selection may cause the tracking path to deviate from the true edge in the early stages, leading to error accumulation and affecting the accuracy and completeness of the overall segmentation result. Especially in the complex scenario addressed in this embodiment, accurate positioning of the starting point can significantly improve the stability and efficiency of subsequent dual-agent collaborative decision-making, thereby ensuring the quality of the segmentation mask generation.

[0017] In this specific implementation, an embodiment employs a method based on a pre-trained target detection model to automatically determine the edge starting point. Specifically, an efficient detection model such as YOLOv5 can be used. First, a global analysis is performed on the input target card / document image to quickly locate the approximate area of ​​the document and the positions of its corner points. After the detection model outputs the coordinates of the four corner points of the document, the system preferentially selects one of them (such as the top left corner) as the starting point for edge tracking. During the training phase, to enhance the robustness of the model, the starting point can also be randomly selected on the edge of the document, but corner points are usually preferred to simulate the natural order of tracking from the corners in actual applications. In this way, the determination of the edge starting point is not only automated but also highly adaptable, capable of handling card / document images with different poses and backgrounds, laying a solid foundation for subsequent edge closure and mask generation.

[0018] S102, starting from the edge starting point, perform multiple rounds of edge tracking on the target card / document image to determine the edge path; wherein, each round of edge tracking includes: taking the current edge point as the center, extracting a local image from the target card / document image, and generating a current state vector based on the local image; inputting the current state vector into a first agent to predict multiple candidate positions of the next edge point; inputting the current state vector into a second agent to predict the offset direction from the current edge point to the next edge point; and determining the next edge point based on the offset direction determined in this round and the multiple candidate positions.

[0019] This embodiment aims to gradually outline the complete and closed contour of a target document from the starting point through continuous and adaptive path tracking. To achieve this goal, this embodiment employs a dual-agent collaborative decision-making framework based on deep reinforcement learning. An agent, in the reinforcement learning paradigm, is an intelligent entity capable of perceiving the environmental state, executing actions, and learning optimal strategies through reward feedback. In each round of edge tracking, the system extracts a local image region from the target document image centered on the current edge point. This is done to focus on local visual information near the current edge point and avoid interference from the complex global background. Based on this local image, the system further generates a state vector that characterizes the current local visual features and the tracking progress. This state vector is then simultaneously input into two agents with different functions: the first agent is responsible for predicting multiple candidate positions for the next edge point, and its output provides various possibilities in the position dimension; the second agent specifically predicts the offset direction from the current point to the next point to capture the local geometric features of the edge direction. By comprehensively considering directional guidance and the distribution of candidate positions, the system can more robustly and accurately determine each subsequent edge point. The core significance of introducing this dual-agent collaborative mechanism lies in decoupling the tasks of location positioning and orientation determination and entrusting them to a dedicated module. Through information complementarity, it effectively addresses noise interference and edge structure changes in complex scenarios, thereby gradually generating a continuous, accurate, and ultimately closure path. This path constitutes the edge contour for selecting the target document, laying a solid foundation for the subsequent generation of a precise segmentation mask.

[0020] In its specific implementation, this embodiment first needs to generate a current state vector for decision-making based on a local image. This process includes using a shared deep convolutional network to extract features from the cropped local image, obtaining an image feature vector containing information such as texture and shape. Simultaneously, the system converts the current edge tracking cycle number (i.e., step size number) into a normalized numerical feature, which reflects the progress information of the tracking process. Finally, by concatenating the image feature vector with this step size feature along the vector dimension, a current state vector integrating visual content and temporal state information is generated. In the specific operation of determining the next edge point, the system first moves the current edge point along the offset direction output by the second agent by a preset number of pixels (e.g., 5 pixels) to obtain a reference position with directional consistency. Subsequently, the system fuses multiple candidate positions output by the first agent with this reference position. Fusion can be achieved by calculating a geometric mean or a weighted average, for example, averaging the coordinates of the candidate positions and the reference position to obtain the final determined coordinates of the next edge point. This fusion strategy effectively combines the diversity of location prediction with the continuity of direction guidance, improving the accuracy and robustness of single-step decision-making.

[0021] In practical applications, both the first and second agents are constructed using a Deep Q-Network (DQN) model. During the training phase, the two agents are not trained independently, but rather collaboratively perform multiple rounds of edge tracking on a large number of labeled sample card and document images, and learn the optimal decision-making strategy by iteratively updating the network parameters. The key signal driving the agents' learning is the training reward. The training rewards for the first and second agents are determined based on two types of rewards generated in each round of edge tracking: dynamic distance reward and orientation consistency reward. The dynamic distance reward is calculated based on the geometric distance (e.g., Euclidean distance) between the agent's predicted next edge point and the actual edge label point. The basic rule is that the closer the predicted point is to the actual edge, the higher the positive reward; conversely, a negative reward or zero reward may be obtained, thus directly guiding the agent to move closer to the actual edge. The orientation consistency reward is determined based on the angle between the offset direction predicted by the second agent and the tangent direction of the actual edge where the current edge point is located. When the angle is less than a certain threshold (e.g., 15 degrees), a positive reward is given, encouraging the agent to ensure that the predicted direction is consistent with the local edge orientation. These two types of rewards together constitute the immediate feedback obtained by the agent through interaction with the "environment" (i.e., images and ground truth labels), which is the core basis for policy optimization in reinforcement learning.

[0022] In the specific training implementation, the dynamic distance reward and orientation consistency reward generated in each round of edge tracking are fused through a weighted summation to form the total training reward used to update the agent in that round. This embodiment further designs an important training mechanism: when calculating the total reward used to update the current decision, the weights of the dynamic distance reward and orientation consistency reward generated in each historical round of edge tracking are not constant, but rather decay as the difference in round number between the corresponding round and the current training round increases. This means that the rewards generated in more distant historical rounds have less influence on the current network parameter update. The main technical effect of introducing this reward weight decay mechanism is that it can effectively mitigate the persistent adverse effects of decision errors or suboptimal reward signals that may arise in early steps on the subsequent long-term training process in long-sequence edge tracking tasks. By making the agent pay more attention to recent and more relevant interaction experiences, this mechanism helps accelerate the convergence speed of the model on complex paths, improves training stability, and ultimately enables the trained model to exhibit stronger anti-error accumulation capabilities and higher overall segmentation accuracy when dealing with long-edge, complex-structured card and ticket images.

[0023] As an example: The training process begins with state initialization: a starting point (preferably the four corner points) is randomly selected from the labeled edges of the sample image, and a 50×50 pixel RGB image patch is cropped centered on this point. This image patch, together with a "step" value normalized to the [0,1] interval representing the current tracking step, constitutes the initial state S0. In each training step (i.e., one round of edge tracking), the system performs the following operations in sequence: First, the current state vector is simultaneously input to the first agent (position agent) and the second agent (orientation agent). The position agent outputs the coordinate offset (Δx1, Δy1), thereby obtaining candidate coordinates based on the current position. The orientation agent outputs a probability distribution of 8 neighborhood directions (up, down, left, right, and four diagonal directions), and selects the direction d with the highest probability as the movement direction. Then, the system executes the action: moving from the current point along direction d by a preset number of pixels (e.g., 5 pixels) to obtain a candidate position. Next, through location fusion strategies (e.g., taking...) and The final position of the next edge point is calculated using the average coordinates. The environment is updated accordingly, with A new image patch is cropped from the center and combined with the updated step number to form the next state. This completes one state transition.

[0024] The reward function provides crucial feedback for the agent's learning. The reward function designed in this embodiment comprises two parts: a dynamic distance reward designed to guide position predictions closer to the true edge, the value of which is given by the formula... Calculate, where D is the predicted point. The Euclidean distance to the true edge, where ε is the scale parameter. If D < 2 pixels, an additional bonus of +1 is awarded; if D < 2 pixels, an additional bonus of +1 is awarded. A penalty of -2 is applied if the predicted direction exceeds the image boundary; a penalty of -1 is applied if the predicted direction fails to approach the true edge for 5 consecutive steps. The directional consistency reward encourages the predicted direction to conform to the edge geometry, and a reward of +0.5 is applied when the angle θ between the predicted direction d and the tangent direction of the true edge at the current point is less than 15°.

[0025] The two agents each have an independent deep Q-network (DQN), whose loss function is constructed based on temporal difference error.

[0026] The loss function of the location agent is:

[0027] The loss function of the orientation agent is:

[0028] in, This represents the Q-value (i.e., the expected cumulative reward) predicted by the network when the location agent chooses action a in state s. This represents the Q-value predicted by the network when the agent chooses action a in state s; , ...represents the immediate reward obtained by the agent in the historical steps back from the current training step (composed of dynamic distance reward and orientation consistency reward). This represents the historical reward decay factor, defined here as λ=0.9. This indicates the degree of reward decay with respect to the current step, where "step" is the step index. This means that earlier steps (with a larger step value) have higher rewards. Weights when calculating the current target Q value ( The smaller the value, the better; This represents the discount factor (0 ≤ γ < 1), used to balance the importance of current immediate rewards and potential future rewards; This indicates the next state that the environment transitions to after action 'a' is performed; This indicates that in the next state s', for all possible actions a', The value is the maximum value. This indicates that in the next state s', for all possible actions a', The value is the maximum value.

[0029] The total loss function for model training is a weighted sum of the losses of the two agents: This weighted sum reflects an optimization approach that prioritizes positional accuracy while using directional consistency as a secondary factor. The model uses the Adam optimizer to minimize the total loss, thereby updating the network parameters. Simultaneously, an empirical replay technique is employed to store and randomly sample training samples, breaking down temporal correlations between data and significantly improving training stability and efficiency.

[0030] S103, construct a segmentation mask based on the determined edge path, and use the segmentation mask to segment the target card document image.

[0031] In this embodiment, the segmentation mask specifically refers to a binary image where the pixel value of the foreground target region (i.e., the document subject) is marked as 255 (white), while the pixel value of the background region is marked as 0 (black). The core significance of constructing the segmentation mask lies in transforming the abstract "edge path" obtained in the previous steps, which consists of a series of ordered coordinate points, into a concrete, pixel-level region identifier map. This mask precisely defines which pixels in the image belong to the document to be extracted, thus enabling it to be directly used to "extract" the document subject from the original complex background. Therefore, constructing a segmentation mask based on the edge path and performing image segmentation accordingly is a key transformation from edge perception to region extraction. The result is a clean image containing only the corrected document subject, providing high-quality input for subsequent high-order tasks such as optical character recognition, and directly determining the final accuracy and reliability of the entire automated processing flow.

[0032] In this specific implementation, the first step is to connect the sequence of edge point coordinates determined in step S102 to form a closed polygonal region. Then, the interior region of this polygon is filled with the foreground (white) in a binary image of the same size, generating an initial segmentation mask. To improve the quality of the mask and make it more suitable for practical applications, this embodiment performs a series of post-mask processing operations. First, morphological optimization is performed. By first dilating the initial mask (using a 3×3 pixel kernel) and then eroding it (also using a 3×3 pixel kernel), small holes and burrs caused by minor edge point jitter or noise can be effectively eliminated, smoothing the segmentation boundaries and making the main body area of ​​the document more complete and coherent. Second, perspective correction is performed. Based on the coordinates of the four corner points of the document detected from the original image or deduced from the path, a homography transformation matrix is ​​calculated. Applying this matrix to the region defined by the mask in the original target card / document image allows the tilted or bent document caused by the shooting angle to be corrected into a regular frontal rectangular view. This step greatly improves the convenience and accuracy of subsequent processing. Finally, OCR input preparation involves inputting the segmented standard document area image, obtained after perspective correction, into an optical character recognition engine (such as PaddleOCR). This allows for the efficient and accurate extraction of various key text fields on the document, such as name, document number, and address, thus completing the entire intelligent processing from raw image to structured information.

[0033] In summary, the method of this embodiment first determines the starting point of the edge of the target card / document image, and then performs multiple rounds of edge tracing from this starting point to gradually construct a complete edge path. In each round of tracing, the system captures a local image centered on the current edge point and extracts features from it to construct a state vector. This state vector is simultaneously input into two parallel agents for collaborative decision-making: the first agent predicts multiple candidate coordinates for the next position, focusing on the probability distribution of spatial location; the second agent focuses on predicting the offset direction from the current point to the next point to capture the local geometric characteristics of the edge direction. By fusing information from both the candidate position and the offset direction, the system can more accurately determine the location of the next edge point. This dual-agent collaborative and information fusion mechanism effectively alleviates three key problems in existing technologies in principle: First, the joint analysis of position candidates and orientation constraints enables mutual verification and optimization of position decisions at each step, significantly improving edge localization accuracy. Second, the state vector dynamically constructed based on local images allows the decision-making process to adaptively focus on the local features of the current edge, thereby enhancing the system's robustness in complex backgrounds or scenarios with changing target poses. Third, thanks to the detailed modeling of the local image context at each step and the fusion of two-dimensional information, the probability of misjudging a single edge point is greatly reduced, effectively preventing the propagation and accumulation of errors in subsequent tracking processes, and ensuring the overall consistency and segmentation accuracy of long-path edge reconstruction. Finally, based on the constructed high-precision edge path, the system can generate a segmentation mask that accurately selects the target region, achieving efficient and stable segmentation of the target region in card and ticket images, significantly improving its segmentation performance in real and complex scenarios.

[0034] In addition, corresponding to Figure 1 The method shown in this embodiment, in another embodiment, also provides a card and ticket image segmentation device. Figure 2 This is a schematic diagram of the structure of the card and document image segmentation device 200, including: The starting point determination module 210 determines the edge starting point of the target card document image.

[0035] The edge tracking module 220 performs multiple rounds of edge tracking on the target card / document image, starting from the edge starting point, to determine the edge path. Each round of edge tracking includes: capturing a local image of the target card / document image centered on the current edge point, and generating a current state vector based on the local image; inputting the current state vector into a first agent to predict multiple candidate positions of the next edge point; inputting the current state vector into a second agent to predict the offset direction from the current edge point to the next edge point; and determining the next edge point based on the offset direction determined in this round and the multiple candidate positions.

[0036] The image segmentation module 230 constructs a segmentation mask based on the determined edge path and uses the segmentation mask to segment the target card document image.

[0037] Optionally, the edge tracking module 220 generates a current state vector based on the local image, including: extracting features from the local image to obtain an image feature vector; and concatenating the image feature vector with a step size feature representing the current round number of edge tracking to generate the current state vector.

[0038] Optionally, the edge tracking module 220 determines the next edge point based on the offset direction determined in the current round and multiple candidate positions, including: moving the current edge point along the offset direction determined in the current round by a preset number of pixels to obtain a reference position; and performing position fusion on the multiple candidate positions determined in the current round and the reference position determined in the current round to obtain the next edge point.

[0039] Optionally, both the first agent and the second agent are based on a deep Q-network model, and during the training phase, the first agent and the second agent collaboratively perform multiple rounds of edge tracking on sample card and document images to update model parameters; wherein, the training rewards for the first agent and the second agent are determined based on the dynamic distance reward and the orientation consistency reward corresponding to each round of edge tracking; the dynamic distance reward is determined based on the geometric distance between the predicted next edge point and the real edge; the orientation consistency reward is determined based on the angle between the predicted offset direction and the tangent direction of the real edge where the current edge point is located.

[0040] Optionally, the training reward is obtained by weighted fusion of the dynamic distance reward and the orientation consistency reward generated in each round of edge tracking; wherein the weights of the dynamic distance reward and the orientation consistency reward in each round of edge tracking decrease as the difference between the round number of the corresponding round and the round number of the current round increases.

[0041] Optionally, the starting point determination module 210 determines the edge starting point of the target card / document image, including: identifying the corner points of the target card / document image based on the target detection model, and determining the corner points as the edge starting points of the target card / document image.

[0042] Optionally, the segmentation mask is used to segment the main body region image of the document from the target card / ticket image, and the main body region image of the document is used to input the optical character recognition model to extract the corresponding document text information.

[0043] It should be noted that the card and document image segmentation device in this embodiment can be used as... Figure 1 The execution body of the method shown is therefore able to achieve... Figure 1 The steps and functions of the method shown are illustrated.

[0044] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Please refer to it. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0045] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0046] Memory is used to store computer programs. Specifically, a computer program may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides the computer program to the processor.

[0047] Specifically, the processor reads the corresponding computer program from non-volatile memory into memory and then runs it, forming the above-mentioned logical structure. Figure 2 The illustrated card / ticket image segmentation device. Correspondingly, the processor executes the program stored in the memory, specifically for performing the following operations: Determine the starting point of the edge of the target card / document image.

[0048] Starting from the edge origin, multiple rounds of edge tracking are performed on the target card / document image to determine the edge path. Each round of edge tracking includes: capturing a local image of the target card / document image centered on the current edge point, and generating a current state vector based on the local image; inputting the current state vector into a first agent to predict multiple candidate positions of the next edge point; inputting the current state vector into a second agent to predict the offset direction from the current edge point to the next edge point; and determining the next edge point based on the offset direction determined in this round and the multiple candidate positions.

[0049] A segmentation mask is constructed based on the determined edge path, and the segmentation mask is used to segment the target card / document image.

[0050] The above is as described in this instruction manual. Figure 1 The card and ticket image segmentation method disclosed in the illustrated embodiment can be applied to a processor and implemented by the processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the processor or by instructions in the form of software. The processor mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0051] Of course, in addition to software implementation, the electronic device described in this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0052] Furthermore, embodiments of this application also propose a computer program product, including a computer-readable storage medium storing one or more computer programs, the one or more computer programs including instructions.

[0053] When the aforementioned instructions are executed by a portable electronic device that includes multiple applications, they enable the portable electronic device to perform... Figure 1The steps in the method shown include: Determine the starting point of the edge of the target card / document image.

[0054] Starting from the edge origin, multiple rounds of edge tracking are performed on the target card / document image to determine the edge path. Each round of edge tracking includes: capturing a local image of the target card / document image centered on the current edge point, and generating a current state vector based on the local image; inputting the current state vector into a first agent to predict multiple candidate positions of the next edge point; inputting the current state vector into a second agent to predict the offset direction from the current edge point to the next edge point; and determining the next edge point based on the offset direction determined in this round and the multiple candidate positions.

[0055] A segmentation mask is constructed based on the determined edge path, and the segmentation mask is used to segment the target card / document image.

[0056] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0057] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0058] The above are merely embodiments of this specification and are not intended to limit the scope of this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification. Furthermore, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this document.

Claims

1. A method for segmenting card and receipt images, characterized in that, include: Determine the starting point of the edge of the target card / document image; Starting from the edge origin, multiple rounds of edge tracking are performed on the target card / document image to determine the edge path. Each round of edge tracking includes: capturing a local image of the target card / document image centered on the current edge point, and generating a current state vector based on the local image; inputting the current state vector into a first agent to predict multiple candidate positions of the next edge point; inputting the current state vector into a second agent to predict the offset direction from the current edge point to the next edge point; and determining the next edge point based on the offset direction determined in this round and the multiple candidate positions. A segmentation mask is constructed based on the determined edge path, and the segmentation mask is used to segment the target card / document image.

2. The method according to claim 1, characterized in that, Generating a current state vector based on the local image includes: Feature extraction is performed on the local image to obtain an image feature vector; The image feature vector is concatenated with the step size feature representing the current round number of edge tracking to generate the current state vector.

3. The method according to claim 1, characterized in that, Based on the offset direction determined in this round and multiple candidate positions, the next edge point is determined, including: Move the current edge point along the offset direction determined in this round by a preset number of pixels to obtain the reference position; The multiple candidate locations determined in this round and the reference location determined in this round are fused to obtain the next edge point.

4. The method according to claim 1, characterized in that, Both the first and second agents are based on a deep Q-network model, and during the training phase, the first and second agents collaboratively perform multiple rounds of edge tracking on sample card and document images to update model parameters. The training rewards for the first and second agents are determined based on the dynamic distance reward and orientation consistency reward corresponding to each round of edge tracking. The dynamic distance reward is determined based on the geometric distance between the predicted next edge point and the real edge. The orientation consistency reward is determined based on the angle between the predicted offset direction and the tangent direction of the real edge where the current edge point is located.

5. The method according to claim 4, characterized in that, The training reward is obtained by weighted fusion of the dynamic distance reward and the orientation consistency reward generated in each round of edge tracking; wherein, the weight of the dynamic distance reward and the orientation consistency reward in each round of edge tracking decreases as the difference between the round number of the corresponding round and the round number of the current round increases.

6. The method according to any one of claims 1 to 5, characterized in that, Determine the starting point of the edge of the target card / document image, including: The target card / document image is identified based on the target detection model, and the corner points are determined as the edge starting points of the target card / document image.

7. The method according to any one of claims 1 to 5, characterized in that, The segmentation mask is used to segment the main body region image of the document from the target card / document image. The main body region image of the document is used as input to the optical character recognition model to extract the corresponding document text information.

8. A card and document image segmentation device, characterized in that, include: The starting point determination module determines the edge starting point of the target card / document image; The edge tracking module performs multiple rounds of edge tracking on the target card / document image, starting from the edge origin, to determine the edge path. Each round of edge tracking includes: capturing a local image of the target card / document image centered on the current edge point, and generating a current state vector based on the local image; inputting the current state vector into a first agent to predict multiple candidate positions for the next edge point; inputting the current state vector into a second agent to predict the offset direction from the current edge point to the next edge point; and determining the next edge point based on the offset direction determined in this round and the multiple candidate positions. The image segmentation module constructs a segmentation mask based on the determined edge paths and uses the segmentation mask to segment the target card / document image.

9. An electronic device, comprising: processor; And a memory arranged to store computer-executable instructions, characterized in that, when executed, the executable instructions cause the processor to perform the method as described in any one of claims 1 to 7.

10. A computer program product, the computer program product comprising a computer-readable storage medium storing a computer program, characterized in that, The computer program is operable to cause the computer to perform the method as described in any one of claims 1 to 7.

Images