Card and document image processing method, system and device and medium

Abstract

The invention discloses a card and document image processing method, system and device and a medium, and relates to the technical field of image processing. Aiming at the problems of perspective distortion, inclined rotation and incorrect direction of a card image, the method comprises the following steps: acquiring a card or document image, and preprocessing the card or document image to generate a standard image; constructing a lightweight multi-task model comprising a lightweight convolutional neural network backbone, a corner regression network branch and a direction classification network branch, and training the lightweight multi-task model; a multi-scale shared feature map is extracted through a lightweight convolutional neural network backbone, then two-dimensional coordinates of four corners are output through a corner regression network branch, and an image direction category is output through a direction classification network branch; constructing a perspective transformation matrix by using the two-dimensional coordinates of the four angular points, and performing perspective distortion correction on the original image; and rotating and outputting the corrected image according to the direction category. According to the invention, the image processing efficiency and accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a card and document image processing method, system, device and medium. BACKGROUND

[0002] With the acceleration of the digitalization process of various online and offline businesses, it has become a common practice for users to take card images such as ID cards, business licenses, and bills by means of mobile phones, scanning devices, etc. However, due to the complexity of the shooting environment, card images often have problems such as perspective distortion, tilt rotation, and incorrect orientation, which seriously affect the accuracy of subsequent text recognition and information extraction.

[0003] In view of the above problems, the academic and industrial circles have proposed various technical solutions for card image geometric correction and orientation determination. However, most of the current solutions still remain in the multi-stage serial process of "corner point detection-perspective correction-independent direction classification". Although the current technology has made certain progress in single tasks such as corner point detection, perspective correction, or direction classification in the field of card and document image processing, there are still the following shortcomings: first, the process is complex and the tasks are scattered, often requiring multiple independent models to be connected in series, resulting in a long overall reasoning time; second, there is a lack of feature sharing between corner point detection and direction determination, which is prone to error accumulation and affects the subsequent recognition accuracy; third, many existing methods have complex network structures and lack support for lightweight and multi-task joint learning, making it difficult to efficiently deploy in different computing environments; fourth, the direction determination granularity is limited, making it difficult to accurately deal with the multi-angle and multi-format card rotation problem in actual shooting scenarios.

[0004] Therefore, it is urgent to propose a technical solution based on a lightweight backbone network, which simultaneously realizes corner point regression and direction classification of card and document images in a single network through a multi-task joint learning mechanism, and has fast and accurate reasoning capability, in order to meet the needs of diversified application scenarios for high precision and high efficiency. SUMMARY

[0005] In view of the needs and deficiencies of the current technology development, the present application provides a card and document image processing method, system, device and medium.

[0006] In a first aspect, the present application provides a card and document image processing method, which adopts the following technical solutions to solve the above technical problems:

[0007] A card and document image processing method, comprising the following steps:

[0008] S1, acquiring a card or document image and preprocessing the acquired image to generate a standard image;

[0009] S2, constructing a lightweight multi-task model, including: a lightweight convolutional neural network backbone based on LCENet, used to extract multi-scale shared feature maps from a standard image; a corner point regression network branch, used to regress the two-dimensional coordinates of the four corner points of the standard image based on the shared feature maps; and a direction classification network branch, used to predict the direction category of the standard image based on the shared feature maps;

[0010] S3, training the lightweight multi-task model using a pre-prepared data set; during the training process: a) introducing perspective perturbation and random rotation strategies to simulate geometric distortion in real scenes and enhance the anti-interference ability of the direction classification network branch to complex poses; b) using a joint loss function, using a smooth L1 loss for the corner point regression branch and a cross-entropy loss for the direction classification branch; c) the joint loss function is a weighted combination of the corner point regression loss and the direction classification loss, and the weight coefficient is dynamically adjusted based on the task error during the training process to realize the cooperative optimization of the two tasks;

[0011] S4, loading the lightweight multi-task model trained in step S3, and inputting the standard image generated in step S1 into the model for inference: extracting multi-scale shared feature maps through the lightweight convolutional neural network backbone based on LCENet, and then outputting the two-dimensional coordinates of the four corner points from the corner point regression network branch and the direction category of the image from the direction classification network branch;

[0012] S5, constructing a perspective transformation matrix using the two-dimensional coordinates of the four corner points regressed by the corner point regression network branch to correct the perspective distortion of the original image; and rotating and outputting the image after perspective distortion correction according to the output result of the direction classification network branch.

[0013] Optionally, the preprocessing operation on the obtained image specifically includes:

[0014] First, the image resolution is uniformly adjusted to a preset size through size normalization;

[0015] Subsequently, mean-variance normalization is performed to eliminate pixel value magnitude differences and adjust the pixel distribution to a standard distribution that meets the subsequent input requirements;

[0016] Finally, a standard image is obtained.

[0017] Optionally, the pre-prepared data set includes images of different types of cards and documents, covering different shooting angles, lighting, and background scenes, wherein each image is labeled with real two-dimensional coordinates of four corner points and a real direction category;

[0018] The images in the data set are subjected to random rotation, scaling, and noise addition data enhancement operations, and are used for training of the lightweight multi-task model.

[0019] Optionally, the corner regression network branch includes a convolutional layer and a fully connected layer, configured to take the shared feature map as input, perform local feature extraction and coordinate information mapping, and output four corner two-dimensional coordinates of the corresponding standard image, thereby providing a position basis for subsequent image perspective correction.

[0020] The direction classification network branch includes a convolutional layer, a global average pooling layer, and a fully connected layer, configured to take the shared feature map as input, perform feature abstraction, global information aggregation, and class mapping, and output a direction class of the corresponding standard image, including 0°, 90°, 180°, and 270°, thereby providing a judgment basis for subsequent image rotation adjustment.

[0021] In a second aspect, the present application provides a card and document image processing system, which solves the above technical problems by adopting the following technical solutions:

[0022] A card and document image processing system is configured to implement the card and document image processing method of the first aspect, and specifically includes:

[0023] An image acquisition module is configured to acquire a card or document image.

[0024] An image preprocessing module is configured to preprocess the acquired image to generate a standard image.

[0025] A model construction module is configured to construct a lightweight multitask model, including: an LCENet-based lightweight convolutional neural network backbone configured to extract a multi-scale shared feature map from the standard image; a corner regression network branch configured to regress two-dimensional coordinates of four corners of the standard image based on the shared feature map; and a direction classification network branch configured to predict a direction class of the standard image based on the shared feature map.

[0026] A model training module is configured to train the lightweight multitask model using a pre-prepared data set. During the training process: a) a perspective disturbance and random rotation strategy is introduced to simulate geometric distortion in a real scene and enhance the anti-interference ability of the direction classification network branch to complex poses; b) a joint loss function is used, a smooth L1 loss is used for the corner regression branch, and a cross-entropy loss is used for the direction classification branch; c) the joint loss function is a weighted combination of the corner regression loss and the direction classification loss, and the weight coefficient is dynamically adjusted based on the task error during the training process to realize the cooperative optimization of the two tasks.

[0027] A loading and calling module is configured to load the trained lightweight multitask model, input the standard image into the model for inference: extract a multi-scale shared feature map based on the LCENet-based lightweight convolutional neural network backbone, and then output two-dimensional coordinates of four corners by the corner regression network branch and output an image direction class by the direction classification network branch.

[0028] The image processing module is configured to firstly regress four corner point two-dimensional coordinates by using the corner point regression network branch to construct a perspective transformation matrix, correct the perspective distortion of the original image, and then rotate the image after the perspective distortion correction according to the output result of the direction classification network branch.

[0029] The image output module is configured to output the card and document image after the perspective distortion correction and the direction correction.

[0030] Optionally, the image preprocessing module specifically comprises:

[0031] The first normalization unit is configured to perform size normalization on the acquired image, and uniformly adjust the image resolution to a preset size.

[0032] The second normalization unit is configured to perform mean-variance normalization on the image after the size normalization, eliminate the pixel value magnitude difference, adjust the pixel distribution to a standard distribution meeting the subsequent input requirement, and obtain a standard image.

[0033] Optionally, the pre-prepared data set comprises images of different types of cards and documents, covers different shooting angles, illuminations and background scenes, wherein each image is labeled with real two-dimensional coordinates of four corner points and a real direction category.

[0034] The images in the data set are subjected to random rotation, scaling and noise addition data enhancement operations, and are used for training of the lightweight multi-task model.

[0035] Optionally, the corner point regression network branch comprises a convolution layer and a fully connected layer, is configured to take the shared feature map as input, perform local feature extraction and coordinate information mapping, and output the two-dimensional coordinates of four corner points of the corresponding standard image, thereby providing a position basis for subsequent image perspective correction.

[0036] The direction classification network branch comprises a convolution layer, a global average pooling layer and a fully connected layer, is configured to take the shared feature map as input, perform feature abstraction, global information aggregation and category mapping, and output the direction category of the corresponding standard image, thereby providing a judgment basis for subsequent image rotation adjustment.

[0037] In a third aspect, the present application provides a card and document image processing device, characterized in that it comprises a memory and at least one processor.

[0038] The memory stores a computer program.

[0039] The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the card and document image processing method according to the first aspect.

[0040] In a fourth aspect, the present application provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program can be executed by a processor to implement the card and document image processing method according to the first aspect.

[0041] Compared with the prior art, the card and document image processing method, system, device and medium of the present application have the following beneficial effects:

[0042] 1. The present application can realize the processing process from the input of the card image to the perspective correction and the direction determination. In the processing process, the multi-scale shared feature map is extracted by using the lightweight convolutional neural network backbone based on LCENet, the two-dimensional coordinates of the four corners of the standard image are regressed by using the corner regression network branch, and the direction category of the standard image is predicted by using the direction classification network branch, which can significantly reduce the inference time delay of the lightweight multi-task model and support the real-time operation of the mobile terminal or embedded device.

[0043] 2. The present application can reduce the negative influence of the corner error on the direction determination, effectively suppress the error cascade phenomenon, improve the geometric correction accuracy and the overall recognition robustness, support the fast processing of various card types and multi-angle shooting images, accurately output the four corner coordinates and the direction label, provide a high-quality preprocessing basis for subsequent OCR and structured recognition, and meet the diversified application needs of government affairs, finance, self-service and the like.

[0044] 3. The present application can realize the corner regression and direction classification in a single model by using the lightweight backbone network, multi-task joint learning and dynamic loss balancing mechanism, and has the following advantages compared with the existing “corner detection—perspective correction—independent direction classification” series method: the inference time is significantly reduced by about 30% on average; the direction classification accuracy is improved by more than 6%; the corner positioning error is reduced by nearly 20%, and the error cascade phenomenon is suppressed; the mobile terminal and embedded edge device can be adapted, and real-time inference can be realized in a low-power consumption environment; a high-quality preprocessing basis can be provided for subsequent OCR and structured recognition, and diversified needs of government affairs, finance and self-service terminals can be met. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a method flowchart of an embodiment of the present application; Figure 1

[0046] Figure 2 is a module connection block diagram of an embodiment of the present application; Figure 2 Figure 3 is a training flowchart of a lightweight multi-task model in an embodiment of the present application.

[0047] Figure 3 DETAILED DESCRIPTION

[0048] ​​​In order to make the technical solutions, the technical problems solved and the technical effects of the present application clearer and more apparent, the technical solutions of the present application are described clearly and completely below in combination with specific embodiments.

[0049] Embodiment one:

[0050] In combination with the accompanying Figure 1 , the present embodiment proposes a card and document image processing method, which includes the following steps:

[0051] S1, acquire a card or document image, and pre-process the acquired image to generate a standard image.

[0052] The card or document image acquired in the present embodiment is not limited to the original image of an ID card, a business license, or a bill acquired through a mobile phone camera or a high-speed scanner.

[0053] The pre-processing operation on the acquired image specifically includes:

[0054] First, the image resolution is uniformly adjusted to a preset size through size normalization;

[0055] Subsequently, mean-variance normalization is performed to eliminate pixel value magnitude differences and adjust the pixel distribution to a standard distribution that meets the subsequent input requirements;

[0056] Finally, a standard image is obtained to adapt to the subsequent lightweight convolutional neural network backbone.

[0057] S2, construct a lightweight multi-task model, including: a lightweight convolutional neural network backbone based on LCENet, used to extract multi-scale shared feature maps from the standard image; an angle point regression network branch, used to regress the two-dimensional coordinates of the four corners of the standard image based on the shared feature maps; a direction classification network branch, used to predict the direction category of the standard image based on the shared feature maps.

[0058] In reference to the accompanying Figure 3 , the lightweight convolutional neural network backbone based on LCENet of the present embodiment includes a stem convolutional layer, Stage1 (low-level features), Stage2 (mid-level features), and Stage3 (high-level semantic features), and its output is a shared feature map. The shared feature map is simultaneously input to the angle point regression network branch and the direction classification network branch.

[0059] The angle point regression network branch of the present embodiment includes a convolutional layer (Conv) and a fully connected layer (FC), which are used to take the shared feature map as input, extract local features, and map coordinate information, and output the two-dimensional coordinates of the four corners of the corresponding standard image, providing a position basis for subsequent image perspective correction.

[0060] The direction classification network branch of the embodiment includes a convolution layer (Conv), a global average pooling layer (GAP) and a fully connected layer (FC), which are used to take the shared feature map as input, output the direction category of the standard image corresponding to the shared feature map through feature abstraction, global information aggregation and category mapping, and the direction category includes four categories of 0°, 90°, 180° and 270°, which provides a judgment basis for subsequent image rotation adjustment.

[0061] S3, training the lightweight multi-task model using a pre-prepared data set; during the training process: a) introducing random rotation (±30°), perspective disturbance (±15%), noise addition and other enhancement operations to simulate geometric distortion in real scenes and enhance the anti-interference ability of the direction classification network branch to complex poses; b) using a joint loss function, using a smooth L1 loss for the corner point regression branch and a cross-entropy loss for the direction classification branch; c) the joint loss function is a weighted combination of the corner point regression loss and the direction classification loss, and the weight coefficient is dynamically adjusted based on the task error during the training process to realize the collaborative optimization of the two tasks.

[0062] The expression of the joint loss function is as follows:

[0063] L_total=Loss_reg+λ*Loss_cls;

[0064] In the formula, Loss_reg represents the corner point regression loss, and Loss_cls represents the direction classification loss.

[0065] During the training process, the weight λ is dynamically adjusted based on the real-time task error to balance the corner point positioning accuracy and the direction classification accuracy and improve the overall robustness of the model.

[0066] S4, loading the lightweight multi-task model trained in step S3, and inputting the standard image generated in step S1 into the model for inference: extracting a multi-scale shared feature map based on the lightweight convolutional neural network backbone of LCENet, and then outputting the two-dimensional coordinates of the four corner points by the corner point regression network branch and outputting the image direction category by the direction classification network branch.

[0067] S5, constructing a perspective transformation matrix using the four corner point two-dimensional coordinates regressed by the corner point regression network branch to correct the perspective distortion of the original image; and rotating the image after the perspective distortion correction according to the output result of the direction classification network branch and outputting the rotated image for subsequent OCR, information extraction or risk control algorithm.

[0068] Experimental results show that, compared with the existing single-task series method, the method of the embodiment improves the direction classification accuracy by more than 6%, reduces the average error of corner point positioning by nearly 20%, and shortens the single-image inference time by about 30%, which can meet the real-time processing needs of mobile terminals and embedded devices.

[0069] Embodiment Two:

[0070] In combination with the accompanying Figure 2 , the embodiment proposes a card and document image processing system, which is suitable for mobile terminal or embedded edge device deployment, and realizes low-latency inference in a computing power limited environment. The system is used to implement the card and document image processing method according to the first aspect, and specifically includes:

[0071] An image acquisition module is configured to acquire a card or document image.

[0072] An image preprocessing module is configured to preprocess the acquired image to generate a standard image.

[0073] A model construction module is configured to construct a lightweight multi-task model, including: a lightweight convolutional neural network backbone based on LCENet, configured to extract multi-scale shared feature maps from the standard image; an angle point regression network branch, configured to regress two-dimensional coordinates of four corner points of the standard image based on the shared feature maps; and a direction classification network branch, configured to predict a direction category of the standard image based on the shared feature maps.

[0074] A model training module is configured to train the lightweight multi-task model using a pre-prepared data set. During the training process: a) a perspective disturbance and random rotation strategy is introduced to simulate geometric distortion in a real scene and enhance the anti-interference ability of the direction classification network branch to complex poses; b) a joint loss function is used, a smooth L1 loss is used for the angle point regression branch, and a cross-entropy loss is used for the direction classification branch; c) the joint loss function is a weighted combination of the angle point regression loss and the direction classification loss, and the weight coefficient is dynamically adjusted based on the task error during the training process to realize the cooperative optimization of the two tasks.

[0075] A loading and calling module is configured to load the trained lightweight multi-task model and input the standard image into the model for inference: multi-scale shared feature maps are extracted through the lightweight convolutional neural network backbone based on LCENet, and then the two-dimensional coordinates of the four corner points are output by the angle point regression network branch, and the direction category of the image is output by the direction classification network branch.

[0076] An image processing module is configured to first construct a perspective transformation matrix using the two-dimensional coordinates of the four corner points regressed by the angle point regression network branch, correct the perspective distortion of the original image, and then rotate the image after the perspective distortion correction according to the output result of the direction classification network branch.

[0077] An image output module is configured to output the card or document image that has completed perspective distortion correction and is in the correct direction.

[0078] In the embodiment, the image preprocessing module specifically includes:

[0079] The normalization unit one is used for size normalization of the obtained image, and image resolution is uniformly adjusted to a preset size;

[0080] The normalization unit two is used for mean-variance normalization of the image after size normalization, eliminating pixel value magnitude difference, and adjusting pixel distribution to a standard distribution meeting subsequent input requirements, to obtain a standard image.

[0081] Reference is made to the accompanying drawings Figure 3 The LCENet-based lightweight convolutional neural network backbone of the embodiment includes a stem convolutional layer, Stage1 (low-layer feature), Stage2 (middle-layer feature), and Stage3 (high-layer semantic feature), and the output is a shared feature map. The shared feature map is simultaneously used as the input of the corner regression network branch and the direction classification network branch.

[0082] The corner regression network branch of the embodiment includes a convolutional layer (Conv) and a fully connected layer (FC), which are used to take the shared feature map as the input, map the coordinate information through local feature extraction, and output the two-dimensional coordinates of four corners of the corresponding standard image, to provide a position basis for subsequent image perspective correction.

[0083] The direction classification network branch of the embodiment includes a convolutional layer (Conv), a global average pooling layer (GAP), and a fully connected layer (FC), which are used to take the shared feature map as the input, map the class through feature abstraction, global information aggregation, and class mapping, and output the direction class of the corresponding standard image, including 0°, 90°, 180°, and 270°, to provide a judgment basis for subsequent image rotation adjustment.

[0084] In the embodiment, the prepared data set includes 100,000+ images of different types of cards and documents, covering different shooting angles, illuminations, and background scenes, wherein each image is labeled with real two-dimensional coordinates of four corners and real direction classes. The images in the data set are subjected to random rotation, scaling, and noise addition data enhancement operations, and are used for training of the lightweight multi-task model.

[0085] The designed joint loss function expression is as follows:

[0086] L_total=Loss_reg+λ*Loss_cls;

[0087] In the formula, Loss_reg represents the corner regression loss, and Loss_cls represents the direction classification loss.

[0088] During the training process, the weight λ is dynamically adjusted based on the real-time task error, to balance the corner positioning accuracy and the direction classification accuracy, and to improve the overall robustness of the model.

[0089] The system described in this embodiment can be deployed and run on mobile terminals or embedded edge devices, and can still achieve an inference time of less than 25 ms for a single image in a computing power limited environment, meeting the real-time application requirements of government affairs, finance and self-service terminals.

[0090] Embodiment three:

[0091] The embodiment provides a card and document image processing device, which is suitable for mobile terminal or embedded edge device deployment and realizes low-delay inference in a computing power limited environment, and specifically comprises a memory and at least one processor.

[0092] The memory stores a computer program.

[0093] The processor can be a CPU, GPU, FPGA, NPU, etc., and is adapted to a mobile terminal SoC and an edge side AI module; the memory can be an eMMC, UFS or other non-volatile storage medium, and supports lightweight model calling. The device can run in an embedded system such as Android and Linux, and is convenient for deployment on a self-service terminal or a mobile device.

[0094] Embodiment four:

[0095] The embodiment provides a computer readable storage medium, which stores a computer program. The computer program can be executed by a processor to realize the card and document image processing method of the embodiment one. The program can be provided by a local installation method, or can be downloaded from a cloud, and supports OTA online updating, so as to optimize model parameters according to a business scenario. The program can run in a lightweight inference framework (such as TensorRT Lite and NCNN), and ensures low power consumption and real-time performance.

[0096] In this case, the program code read from the storage medium itself can realize the functions of any one of the above embodiments, so the program code and the storage medium storing the program code constitute a part of the present application.

[0097] The storage medium for providing the program code includes a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card and a ROM. Alternatively, the program code can be downloaded from a server computer through a communication network.

[0098] In addition, it should be clear that not only the program code read by the computer can be executed, but also part or all of the actual operations can be completed by the operating system and the like operating on the computer based on the instructions of the program code, so as to realize the functions of any one of the above embodiments.

[0099] Further, it is understood that the program code read by the storage medium can be written into the memory provided in the expansion board inserted into the computer or the memory provided in the expansion unit connected to the computer, and then the CPU or the like mounted on the expansion board or the expansion unit is caused to perform part or all of the actual operation based on the instruction of the program code, thereby realizing the function of any of the above-described embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can still be modified, or part or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for processing image data of cards and documents, characterized in that, Includes the following steps: S1. Obtain the image of the card or document, and preprocess the obtained image to generate a standard image; S2. Construct a lightweight multi-task model, including: a lightweight convolutional neural network backbone based on LCENet, used to extract multi-scale shared feature maps from standard images; a corner regression network branch, used to regress the two-dimensional coordinates of the four corner points of the standard image based on the shared feature maps; and an orientation classification network branch, used to predict the orientation category of the standard image based on the shared feature maps. S3. Train a lightweight multi-task model using a pre-prepared dataset. During training: a) Introduce perspective perturbation and random rotation strategies to simulate geometric distortions in real-world scenes and enhance the anti-interference ability of the orientation classification network branch against complex poses; b) Employ a joint loss function, using smooth L1 loss for the corner regression branch and cross-entropy loss for the orientation classification branch; c) The joint loss function is a weighted combination of corner regression loss and orientation classification loss, with the weight coefficients dynamically adjusted based on task errors during training to achieve collaborative optimization of the two tasks. S4. Load the lightweight multi-task model trained in step S3, and input the standard image generated in step S1 into the model for inference: extract multi-scale shared feature maps through the lightweight convolutional neural network backbone based on LCENet, and then output the two-dimensional coordinates of the four corner points by the corner regression network branch and the direction classification network branch outputs the image direction category. S5. Construct a perspective transformation matrix using the two-dimensional coordinates of the four corner points from the corner regression network branch regression, and correct perspective distortion in the original image; rotate and output the image after perspective distortion correction based on the output of the orientation classification network branch.

2. The card and document image processing method according to claim 1, characterized in that, The preprocessing operations performed on the acquired images specifically include: First, the image resolution is uniformly adjusted to the preset size through size normalization; Subsequently, mean and variance normalization is performed to eliminate differences in pixel value magnitudes and adjust the pixel distribution to a standard distribution that meets the requirements of subsequent input. The final result is a standard image.

3. The card and document image processing method according to claim 1, characterized in that, The pre-prepared dataset includes images of different types of cards and documents, covering different shooting angles, lighting, and background scenes. Each image is labeled with the real two-dimensional coordinates of the four corner points and the real direction category. We perform random rotation, scaling, and noise augmentation on the images in the dataset and use them to train a lightweight multi-task model.

4. The card and document image processing method according to claim 1, characterized in that, The corner regression network branch includes convolutional layers and fully connected layers, which are used to take the shared feature map as input, extract local features and map coordinate information, and output the two-dimensional coordinates of the four corner points of the corresponding standard image, so as to provide a positional basis for subsequent image perspective correction. The orientation classification network branch includes convolutional layers, global average pooling layers, and fully connected layers. It takes a shared feature map as input and outputs the orientation category of the corresponding standard image through feature abstraction, global information aggregation, and category mapping. The categories include 0°, 90°, 180°, and 270°, providing a basis for subsequent image rotation adjustments.

5. A card and document image processing system, characterized in that, It is used to implement the card and document image processing method as described in any one of claims 1-4, specifically including: Image acquisition module, used to acquire images of cards or documents; The image preprocessing module is used to preprocess the acquired images and generate standard images; The model building module is used to build lightweight multi-task models, including: a lightweight convolutional neural network backbone based on LCENet, used to extract multi-scale shared feature maps from standard images; a corner regression network branch, used to regress the two-dimensional coordinates of the four corner points of the standard image based on the shared feature maps; and an orientation classification network branch, used to predict the orientation category of the standard image based on the shared feature maps. The model training module is used to train a lightweight multi-task model using a pre-prepared dataset. During training: a) perspective perturbation and random rotation strategies are introduced to simulate geometric distortions in real-world scenes, enhancing the anti-interference ability of the orientation classification network branch against complex poses; b) a joint loss function is adopted, using smooth L1 loss for the corner regression branch and cross-entropy loss for the orientation classification branch; c) the joint loss function is a weighted combination of corner regression loss and orientation classification loss, and the weight coefficients are dynamically adjusted based on task errors during training to achieve collaborative optimization of the two tasks. The loading and calling module is used to load the trained lightweight multi-task model and input the standard image into the model for inference: multi-scale shared feature maps are extracted through the lightweight convolutional neural network backbone based on LCENet, and then the corner regression network branch outputs the two-dimensional coordinates of the four corner points, and the direction classification network branch outputs the image direction category. The image processing module first constructs a perspective transformation matrix using the two-dimensional coordinates of the four corner points regressed by the corner regression network branch, corrects perspective distortion in the original image, and then rotates the perspective-distortion-corrected image based on the output of the orientation classification network branch. The image output module is used to output images of cards or documents that have been corrected for perspective distortion and are oriented correctly.

6. The card and document image processing system according to claim 5, characterized in that, The image preprocessing module specifically includes: Normalization unit one is used to normalize the size of the acquired image and adjust the image resolution to a preset size. Normalization unit two is used to normalize the mean and variance of the image after size normalization, eliminate the difference in pixel value magnitude, adjust the pixel distribution to a standard distribution that meets the requirements of subsequent input, and obtain a standard image.

7. The card and document image processing system according to claim 5, characterized in that, The pre-prepared dataset includes images of different types of cards and documents, covering different shooting angles, lighting, and background scenes. Each image is labeled with the real two-dimensional coordinates of the four corner points and the real direction category. We perform random rotation, scaling, and noise augmentation on the images in the dataset and use them to train a lightweight multi-task model.

8. The card and document image processing system according to claim 5, characterized in that, The corner regression network branch includes convolutional layers and fully connected layers, which are used to take the shared feature map as input, extract local features and map coordinate information, and output the two-dimensional coordinates of the four corner points of the corresponding standard image, so as to provide a positional basis for subsequent image perspective correction. The orientation classification network branch includes convolutional layers, global average pooling layers, and fully connected layers. It takes a shared feature map as input and outputs the orientation category of the corresponding standard image through feature abstraction, global information aggregation, and category mapping. The categories include 0°, 90°, 180°, and 270°, providing a basis for subsequent image rotation adjustments.

9. A card and document image processing device, characterized in that, include: Memory and at least one processor; The memory contains computer programs; The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the card and document image processing method as described in any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the card and document image processing method as described in any one of claims 1-4.

Images