Automatic SIM (Subscriber Identity Module) card identification method based on semantic segmentation and shape analysis

Abstract

The invention provides an SIM card automatic identification method based on semantic segmentation and shape analysis, and the method comprises the steps: inputting collected image information containing an SIM card into a semantic segmentation model, classifying the image pixel level as the SIM card or background, and generating a probability graph; performing threshold processing on the probability graph to generate a binary image; marking and extracting a connected region in the binary image by using a connected region analysis technology, and identifying an SIM card region; calculating a minimum enclosing rectangle for each identified SIM card connected domain, calculating a required rotation angle according to the angle of the minimum enclosing rectangle, and correcting the direction of the SIM card area by using affine transformation or perspective transformation; inputting the corrected SIM card image into an OCR (Optical Character Recognition) model, and carrying out character segmentation and recognition; and converting an identification result of the OCR model into a structured data format and outputting the structured data format. The method has significant application value in refined management scenes such as batch checking and inventory management of SIM cards, and can significantly improve the automation level and work efficiency of SIM card management.

Description

Technical Field

[0001] The invention relates to the technical field of smart card identification, and in particular to a SIM card automatic identification method based on semantic segmentation and shape analysis. Background Art

[0002] In the current technical field, automatic identification of SIM card areas is an important application in the field of image processing and computer vision. It is widely used in multiple scenarios such as mobile communications, smart device identification, and automated processing. Existing technical solutions mainly rely on a series of image processing algorithms to achieve this goal. These algorithms include but are not limited to classic methods such as edge detection, morphological processing, and connected domain analysis. In order to improve the processing effect, these algorithms usually perform a series of preprocessing steps on the original image before application, such as graying to simplify image information, filtering to remove noise, and binarization to clarify the boundary between foreground and background.

[0003] Although the above technical solution can realize automatic identification of SIM card area to a certain extent, it still faces many challenges and limitations in practical application:

[0004] Strict image quality and clarity requirements: Existing methods are highly dependent on image quality and clarity. When the image is disturbed by noise, light changes, or there are scratches and stains on the surface of the SIM card, the recognition accuracy will drop significantly. Especially in low light conditions or scenes where the SIM card is partially blocked, the algorithm is not robust enough to ensure stable recognition performance.

[0005] Poor shape and angle adaptability: SIM cards may present a variety of shapes and angles in actual use. However, existing methods are often unable to handle these non-standard shapes. In order to accurately identify, it is usually necessary to perform complex pre-processing on the image in advance, such as rotation correction and shape regularization, which not only increases the complexity of processing, but also limits the versatility and practicality of the algorithm.

[0006] High demand for manual intervention: When identifying the extracted SIM card area, existing methods often cannot directly output accurate results that meet the requirements of subsequent processing. Therefore, manual intervention is often required for further correction and verification, which not only increases the workload, but also increases the processing cost and reduces the degree of automation of the overall process.

[0007] In addition, with the rapid development of deep learning technology, although there have been attempts to use deep learning models such as convolutional neural networks (CNN) for image classification to identify SIM card areas, these methods often require a large amount of data for training, and for SIM card images with varying shapes and angles, their generalization ability and recognition accuracy still need to be improved.

[0008] In summary, the existing technology still has many shortcomings in the automatic identification of SIM card areas, especially in processing complex image environments, non-standard SIM cards and reducing human intervention. New technical solutions are urgently needed to break through these limitations and improve the accuracy and efficiency of identification. Summary of the invention

[0009] In view of the shortcomings of the prior art, the present invention provides a SIM card automatic recognition method based on semantic segmentation and shape analysis, aiming to improve recognition precision and accuracy, enhance adaptability, and reduce manual intervention, and provide an integrated solution from image segmentation, SIM card area extraction to sequence recognition, which has significant application value in refined management scenarios such as batch inventory and inventory management of SIM cards, and can significantly improve the automation level and work efficiency of SIM card management.

[0010] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0011] A SIM card automatic recognition method based on semantic segmentation and shape analysis comprises the following steps:

[0012] Collect image information including SIM card;

[0013] Input the collected image information into a pre-trained semantic segmentation model, which classifies the image pixel level into SIM card or background and generates a probability map; wherein the probability map represents the probability of each pixel belonging to the SIM card or the background;

[0014] Performing threshold processing on the probability map to generate a binary image, wherein the pixel value of the SIM card area is 1 and the pixel value of the background area is 0;

[0015] Using connected domain analysis technology to mark and extract connected areas in the binary image, and identify the SIM card area;

[0016] Calculate the minimum bounding rectangle for each identified SIM card connected domain, calculate the required rotation angle according to the angle of the minimum bounding rectangle, and use affine transformation or perspective transformation to correct the direction of the SIM card area to ensure that it is horizontal or vertical;

[0017] Input the corrected SIM card image into the OCR model for character segmentation and recognition;

[0018] The recognition result of the OCR model is converted into a structured data format and outputted.

[0019] According to a SIM card automatic identification method based on semantic segmentation and shape analysis provided by the present invention, a camera is used to collect image information containing a SIM card, and image collection of a single SIM card and multiple randomly placed loose cards is supported, and the image resolution is set to 1080p;

[0020] The image acquisition process is represented by the function I = f capture (resolution), resolution is the image resolution parameter, which is used to specify the resolution of the captured image.

[0021] According to a SIM card automatic identification method based on semantic segmentation and shape analysis provided by the present invention, the step of inputting the collected image information into a pre-trained semantic segmentation model comprises:

[0022] Use the pre-trained semantic segmentation model DeepLabV3+ to process the image;

[0023] The DeepLabV3+ model uses dilated convolution and spatial pyramid pooling ASPP technology to extract multi-scale features in images;

[0024] The model training strategy of the DeepLabV3+ model uses transfer learning to apply the pre-trained model weights to a SIM card-specific image dataset to accelerate the training process and improve the accuracy and efficiency of the model in SIM card recognition.

[0025] During model training, a large number of image datasets containing SIM cards of various shapes, angles, and partial occlusions are used to improve the generalization and adaptability of the model;

[0026] Among them, the semantic segmentation process is expressed as S = f seg (I,θ), where S is the segmented image, θ is the parameter of the model, and f seg is a semantic segmentation function, where the input image I is processed by the model parameters θ.

[0027] According to a SIM card automatic recognition method based on semantic segmentation and shape analysis provided by the present invention, the construction of the semantic segmentation model DeepLabV3+ specifically includes:

[0028] Building the model architecture: Using a pre-trained convolutional neural network as the backbone network to extract high-level features of the image, and combining it with the ASPP module with dilated convolutions with different dilation rates to capture multi-scale contextual information;

[0029] Implement dilated convolution: insert holes between the elements of the convolution kernel to expand the receptive field and capture a wider range of contextual information in the image while keeping the computational complexity unchanged;

[0030] Applying spatial pyramid pooling: Integrate a global average pooling layer in the ASPP module to extract global features of the image and fuse them with multi-scale feature maps to enhance the model’s understanding of the overall structure and context of the image;

[0031] Add decoder module: By introducing a decoder module, the output of the ASPP module is upsampled to the same resolution as the input image using a deconvolution or upsampling layer to refine the segmentation result;

[0032] Execute a transfer learning training strategy: Initialize the model using pre-trained model weights and fine-tune it on a SIM card-specific image dataset so that the model learns the mapping relationship that classifies the image pixel level as SIM card or background;

[0033] Generate output: The model outputs a probability map where the value of each pixel represents the probability of the pixel belonging to the SIM card or the background.

[0034] According to a SIM card automatic recognition method based on semantic segmentation and shape analysis provided by the present invention, before the connected domain analysis, threshold processing is performed on the probability map output by the semantic segmentation model, specifically including:

[0035] Determine a threshold T according to the SIM card image characteristics, and this threshold is used to classify the pixel points in the probability map as SIM card or background;

[0036] Compare each pixel value in the probability map with the determined threshold T one by one. If the pixel value is greater than or equal to T, the pixel is classified as a SIM card and marked as 1; if the pixel value is less than T, the pixel is classified as background and marked as 0;

[0037] Based on the comparison results, a new binary image is generated, in which the pixels marked as 1 constitute the SIM card area, and the pixels marked as 0 constitute the background area.

[0038] According to a SIM card automatic identification method based on semantic segmentation and shape analysis provided by the present invention, the connected domain analysis is expressed as the following formula:

[0039] C = f cca (S,T)

[0040] Among them, C is the connected domain set, T is the threshold parameter, and f cca is the connected domain analysis function.

[0041] According to a SIM card automatic identification method based on semantic segmentation and shape analysis provided by the present invention, the correction of the minimum circumscribed rectangle is expressed as the following formula:

[0042] R = f rect (C,α)

[0043] Among them, R is the corrected rectangular area, α is the rotation angle, and f rect is the minimum bounding rectangle correction function.

[0044] According to a SIM card automatic recognition method based on semantic segmentation and shape analysis provided by the present invention, when performing character segmentation, a projection segmentation method or an edge detection segmentation method is used to accurately segment a continuous character sequence in an image into single characters; wherein the projection segmentation method determines the row and column boundaries of the characters by calculating the horizontal projection and vertical projection of the image, and the edge detection segmentation method uses an edge detection algorithm to recognize the outline of the character to achieve segmentation.

[0045] According to a SIM card automatic recognition method based on semantic segmentation and shape analysis provided by the present invention, when performing character recognition, a deep learning model combining a convolutional neural network, a long short-term memory network and a connection time series classification is used to recognize the segmented single character image;

[0046] Use convolutional neural networks to extract features from the segmented single character images; input the extracted features into long short-term memory networks to capture the temporal dependencies in the character sequence and perform sequence modeling; use connection temporal classification as a loss function to train the model and handle the problem of mismatching the lengths of input and output sequences, and directly output the recognized character sequence, thereby achieving accurate segmentation and recognition of the characters on the SIM card;

[0047] The output character sequence is highly consistent with the actual character sequence on the SIM card.

[0048] According to a SIM card automatic recognition method based on semantic segmentation and shape analysis provided by the present invention, before the collected image information is input into the semantic segmentation model, or before the probability map output by the semantic segmentation model is threshold processed, local features are first extracted from the image or the probability map;

[0049] Based on the extracted local features, an adaptive threshold calculation algorithm is used to dynamically calculate the threshold corresponding to each area or pixel according to the local feature differences of different areas in the image or probability map;

[0050] The calculated adaptive threshold is used to perform threshold processing on the image or probability map to generate a binary image. During the processing, each pixel or area is judged according to its corresponding adaptive threshold, and the pixel values ​​above the threshold are classified as the SIM card area, and the pixel values ​​below the threshold are classified as the background.

[0051] It can be seen that compared with the prior art, the present invention proposes an innovative SIM card region automatic identification method, which shows significant advantages and many beneficial effects, which are specifically reflected in the following aspects:

[0052] 1. Significantly improve recognition accuracy: The present invention adopts advanced semantic segmentation technology to achieve fine classification of image pixels, and can accurately distinguish the SIM card area from the background. Compared with the existing methods based on edge detection and morphological processing, the present invention has a higher recognition accuracy rate when processing complex image environments, partially blocked or poor quality SIM card images, ensuring the accuracy and reliability of the recognition results.

[0053] 2. Enhanced adaptability: By training a highly generalized semantic segmentation model, the present invention can adaptively process SIM card images of various shapes, angles, and partial occlusions without complex preprocessing steps or tedious parameter adjustments. This feature greatly broadens the application scope of the present invention, improves its practicality and flexibility, and enables it to easily cope with a variety of practical application scenarios.

[0054] 3. Effectively reduce manual intervention: The present invention cleverly combines technologies such as connected domain analysis and minimum circumscribed rectangle to achieve automatic correction and direction adjustment of the SIM card area, significantly reducing the manual intervention and correction workload in subsequent processing. This improvement not only reduces labor costs, but also improves the degree of automation of the entire recognition process and improves work efficiency.

[0055] 4. Significantly improve sequence recognition accuracy: After accurately identifying and extracting the SIM card area, the present invention further inputs the corrected image into the optimized OCR model to recognize the number / letter sequence, thereby achieving a higher sequence recognition accuracy. The addition of this step significantly improves the accuracy of sequence recognition in the present invention compared to existing methods, providing a more accurate data basis for subsequent information processing and application.

[0056] 5. Provide an integrated solution: The present invention integrates multiple steps such as image segmentation, SIM card area extraction, direction correction and sequence recognition into an integrated solution, which simplifies the processing flow, reduces the error accumulation in the intermediate links, and improves the efficiency and accuracy of the overall processing. Compared with the existing solutions that require multiple steps and method combinations, the present invention is more concise, efficient, easy to implement and maintain.

[0057] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The present invention is a flowchart of an embodiment of a method for automatic identification of a SIM card based on semantic segmentation and shape analysis. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0061] See also Figure 1 This embodiment provides a method for automatic identification of a SIM card based on semantic segmentation and shape analysis, the method comprising the following steps:

[0062] Step S1, collecting image information containing the SIM card. The image information containing the SIM card is collected using a camera, supporting image collection of a single SIM card and multiple randomly placed loose cards, and setting the image resolution to 1080p.

[0063] Step S2, input the collected image information into a pre-trained semantic segmentation model, the semantic segmentation model classifies the image pixel level into SIM card or background, and generates a probability map; wherein the probability map represents the probability of each pixel belonging to the SIM card or the background. It can be seen that this embodiment can automatically identify the SIM card area in the image by training the model without complex preprocessing and parameter adjustment. At the same time, since the semantic segmentation technology can process SIM cards of any shape, angle and partial occlusion, it can improve the impact of image quality and clarity on recognition accuracy.

[0064] Step S3, threshold processing is performed on the probability map to generate a binary map, in which the pixel value of the SIM card area is 1 and the pixel value of the background area is 0.

[0065] Step S4, using the connected domain analysis technology to mark and extract the connected regions in the binary image, and identify the SIM card region. It can be seen that this embodiment finds the SIM card region through the connected domain analysis. The connected domain analysis is a commonly used image processing technology that can effectively identify the connected regions in the image, thereby accurately extracting the SIM card region.

[0066] Step S5, calculate the minimum bounding rectangle for each identified SIM card connected domain, calculate the required rotation angle according to the angle of the minimum bounding rectangle, and use affine transformation or perspective transformation to correct the direction of the SIM card area to ensure that it is horizontal or vertical. It can be seen that this embodiment uses the minimum bounding rectangle and other technologies to correct the direction of the SIM card area, thereby automatically correcting the direction of the SIM card area. Among them, the minimum bounding rectangle is a commonly used geometric shape calculation method, which can effectively determine the minimum rectangle surrounding the SIM card area, thereby accurately correcting the direction of the SIM card area.

[0067] Step S6, input the corrected SIM card image into the OCR model for character segmentation and recognition. It can be seen that a large number of image data sets containing SIM card number / letter sequences are used in the model training process. In this embodiment, the corrected SIM card image is input into the OCR model, and the output of the model is the corresponding number / letter sequence. The OCR model can effectively recognize the number and letter sequence in the image, thereby realizing automatic recognition of the SIM card sequence.

[0068] Step S7, convert the recognition results of the OCR model into a structured data format for output, which is convenient for subsequent management and application. The recognition results usually include sequences of numbers and letters, which need to be formatted into the data structure required by a specific application, such as JSON or XML. In addition, error detection and correction are required to ensure the accuracy of the data.

[0069] Therefore, through the above technical means, this embodiment can realize automatic recognition and extraction of SIM cards of any shape, angle and partial obstruction, thereby improving the SIM card extraction accuracy and sequence recognition accuracy, and reducing manual intervention and costs.

[0070] In the above step S1, the image acquisition process is represented by the function I = f capture (redolution), resolution is the image resolution parameter, which is used to specify the resolution of the acquired image.

[0071] In the above step S2, the collected image information is input into a pre-trained semantic segmentation model, including:

[0072] The images were processed using the pre-trained semantic segmentation model DeepLabV3+; the DeepLabV3+ model used dilated convolution and spatial pyramid pooling ASPP technology to extract multi-scale features in the image; the DeepLabV3+ model's model training strategy adopted transfer learning, applying the pre-trained model weights to a SIM card-specific image dataset to accelerate the training process and improve the model's accuracy and efficiency in SIM card recognition; during the model training process, a large number of image datasets containing SIM cards of various shapes, angles, and partial occlusions were used to improve the model's generalization and adaptability. The output of the model is a probability map, indicating the probability of each pixel belonging to a SIM card or background.

[0073] In the above step S2, the semantic segmentation process is expressed as S = f seg (I,θ), where S is the segmented image, θ is the parameter of the model, and f seg is a semantic segmentation function, where the input image I is processed by the model parameters θ.

[0074] In this embodiment, the construction of the semantic segmentation model DeepLabV3+ specifically includes:

[0075] Model architecture: A pre-trained convolutional neural network is used as the backbone network to extract high-level features of the image, and the ASPP module with dilated convolutions with different dilation rates is combined to capture multi-scale contextual information. The ASPP module is implemented by multiple convolution branches in parallel, each with a different dilation rate to capture features at different scales.

[0076] Implementing dilated convolution: Insert holes between the elements of the convolution kernel to expand the receptive field, and capture a wider range of contextual information in the image while keeping the computational complexity unchanged. Among them, dilated convolution is a special convolution operation that expands the receptive field of the convolution kernel by inserting holes (i.e., unfilled pixels) between the elements of the convolution kernel. This can capture a wider range of contextual information without increasing the computational complexity.

[0077] Applying spatial pyramid pooling: A global average pooling layer is integrated in the ASPP module to extract global features of the image and fuse them with multi-scale feature maps, thereby enhancing the model's understanding of the overall structure and context of the image. It can be seen that the ASPP module of this embodiment also includes a global average pooling layer, which extracts global features of the image and fuses these features with multi-scale feature maps. This fusion helps the model better understand the overall structure and context of the image.

[0078] Add decoder module: By introducing a decoder module, the output of the ASPP module is upsampled to the same resolution as the input image using a deconvolution or upsampling layer to refine the segmentation result. It can be seen that DeepLabV3+ refines the segmentation result by adding a simple decoder module. The decoder module usually includes a series of deconvolution or upsampling layers to upsample the output of the ASPP module to the same resolution as the input image.

[0079] Execute transfer learning training strategy: The model is trained using a transfer learning strategy, which is to initialize the model with pre-trained model weights and fine-tune it on a SIM card-specific image dataset. During the training process, the model learns the mapping relationship that classifies the image pixel level as SIM card or background.

[0080] Generate output: The model outputs a probability map, where the value of each pixel represents the probability of the pixel belonging to the SIM card or the background. By setting a threshold, the probability map can be converted into a binary map, thereby separating the SIM card area from the background.

[0081] Through the above construction process, the DeepLabV3+ model can effectively separate the SIM card from the background in the image and provide accurate segmentation results for subsequent shape analysis and OCR recognition. This method not only improves the impact of image quality and clarity on recognition accuracy, but also avoids complex preprocessing and parameter adjustment, improving the universality and practicality of the method.

[0082] In the above step S4, the probability map output by the semantic segmentation model is thresholded, and pixels with probabilities higher than a certain threshold are regarded as part of the SIM card. Then, a connected domain analysis algorithm, such as a two-pass scanning method or a graph-based method, is applied to mark and extract connected regions, thereby extracting the SIM card region. Connected domain analysis can effectively identify connected regions in an image, thereby accurately extracting the SIM card region.

[0083] In this embodiment, the connected domain analysis is expressed as the following formula:

[0084] C = f cca (S,T)

[0085] Among them, C is the connected domain set, T is the threshold parameter, and f cca is the connected domain analysis function.

[0086] Specifically, before performing connected domain analysis, the probability map output by the semantic segmentation model DeepLabV3+ needs to be thresholded. This process involves converting the probability map into a binary map, where the pixel value of the SIM card area is 1 and the pixel value of the background area is 0. Thresholding is a key step in image segmentation, which determines which pixels are classified as SIM cards and which are classified as background. The specific steps of thresholding are as follows:

[0087] Determine the threshold: Determine a threshold T according to the characteristics of the SIM card image. This threshold is used to classify the pixels in the probability map as SIM card or background.

[0088] Apply threshold: Compare each pixel value in the probability map with the determined threshold T one by one. If the pixel value is greater than or equal to T, the pixel is classified as a SIM card and marked as 1; if the pixel value is less than T, the pixel is classified as background and marked as 0;

[0089] Generate binary image: Generate a new binary image based on the comparison result, in which the pixels marked as 1 constitute the SIM card area, and the pixels marked as 0 constitute the background area.

[0090] In the above step S5, the correction of the minimum circumscribed rectangle is expressed as the following formula:

[0091] R = f rect (C,α)

[0092] Among them, R is the corrected rectangular area, α is the rotation angle, and f rect is the minimum bounding rectangle correction function.

[0093] In the above step S6, OCR recognition can be expressed as O = f ocr (R,φ), where O is the recognition result, φ represents the OCR model parameters, and f ocr It is the OCR recognition function.

[0094] Specifically, character segmentation technology is the process of segmenting a continuous sequence of characters in an image into individual characters. This is crucial for subsequent character recognition because each character needs to be recognized individually. The methods used for character segmentation include:

[0095] Projection segmentation: includes horizontal and vertical projection, which are used to determine the boundaries of characters. Horizontal projection: By calculating the sum of pixels in each row of the image, the row boundaries of characters can be determined; vertical projection: By calculating the sum of pixels in each column of the image, the column boundaries of characters can be determined.

[0096] Edge detection segmentation: Use edge detection algorithms (such as Canny edge detection) to identify the outline of characters and segment the characters accordingly.

[0097] Therefore, when performing character segmentation, this embodiment preferably adopts projection segmentation or edge detection segmentation technology to accurately segment the continuous character sequence in the image into single characters; among them, the projection segmentation method determines the row and column boundaries of the characters by calculating the horizontal and vertical projections of the image, and the edge detection method segmentation method uses the edge detection algorithm to identify the contours of the characters to achieve segmentation.

[0098] Character recognition is the process of converting a segmented single character image into a corresponding character or number. When performing character recognition, this embodiment uses a deep learning model that combines a convolutional neural network (CNN), a long short-term memory network (LSTM), and a connection temporal classification (CTC). This model can handle the problem of sequential recognition of text lines. The specific steps are as follows:

[0099] Feature extraction: Use convolutional neural network (CNN) to extract features of text images;

[0100] Sequence modeling: The features extracted by the convolutional neural network (CNN) are input into the long short-term memory (LSTM) network, which can capture the temporal dependencies in the sequence.

[0101] Sequence recognition: The model is trained using the Connectionist Temporal Classification (CTC) loss function. Connectionist Temporal Classification (CTC) can handle the problem of mismatched input and output sequence lengths and directly output the recognized character sequence.

[0102] Therefore, this embodiment uses a convolutional neural network to extract features from the segmented single character image; the extracted features are input into a long short-term memory network to capture the time dependency in the character sequence and perform sequence modeling; the connection temporal classification is used as a loss function to train the model and handle the problem of mismatch between the input and output sequence lengths, and the recognized character sequence is directly output, thereby achieving accurate segmentation and recognition of the characters on the SIM card; wherein the output character sequence is highly consistent with the actual character sequence on the SIM card.

[0103] Although character segmentation and recognition technologies are conventional technologies in the field of OCR, these technologies have been widely studied and applied, and many commercial and open source OCR software have integrated these technologies. However, in this embodiment, the training process of the OCR model uses a large number of image data sets containing SIM card number / letter sequences, and the output of the model is the corresponding number / letter sequence, which realizes the automatic recognition of the SIM card sequence. Therefore, although character segmentation and recognition are conventional technologies, their application in specific application scenarios (such as SIM card recognition) is still innovative and can improve the accuracy and efficiency of recognition.

[0104] Of course, for the character recognition process, this embodiment may also use algorithms such as beam search to improve the accuracy and efficiency of recognition.

[0105] In this embodiment, before the collected image information is input into the semantic segmentation model, or before the probability map output by the semantic segmentation model is thresholded, local features are first extracted from the image or probability map, wherein the local features include but are not limited to pixel brightness, contrast, texture information, edge information, etc.

[0106] Based on the extracted local features, an adaptive threshold calculation algorithm is used to dynamically calculate the threshold corresponding to each region or pixel according to the local feature differences of different regions in the image or probability map. For example, for areas with high brightness and obvious contrast, a higher threshold can be selected to better separate the foreground and background; while for areas with low brightness or unclear contrast, a lower threshold can be selected to avoid mis-segmentation.

[0107] The calculated adaptive threshold is used to perform threshold processing on the image or probability map to generate a binary image. During the processing, each pixel or area is judged according to its corresponding adaptive threshold, and the pixel values ​​above the threshold are classified as the SIM card area, and the pixel values ​​below the threshold are classified as the background.

[0108] The generated binary image is further optimized, such as removing isolated points, filling small holes, smoothing edges, etc., to obtain a more accurate and clear binary image of the SIM card area.

[0109] Therefore, through the above-mentioned adaptive threshold processing step, this embodiment provides a method for dynamically adjusting the threshold according to the local features of the image, which can more effectively handle the SIM card image segmentation problem under complex conditions such as different lighting conditions and contrast differences, and improve the accuracy of the binary image and the reliability of subsequent recognition.

[0110] In this embodiment, for the problem of high requirements for image quality and clarity: the existing methods based on image processing and deep learning will significantly reduce the recognition accuracy when processing poor image quality or partial occlusion of the SIM card. This embodiment uses semantic segmentation technology to adaptively process SIM cards of any shape, angle and partial occlusion, thereby improving the impact of image quality and clarity on recognition accuracy.

[0111] Regarding the problem that complex preprocessing and parameter adjustment are required to process SIM cards of different shapes and angles: Existing methods often require complex preprocessing and parameter adjustment when processing SIM cards of different shapes and angles, which limits their application scope and practicality. This embodiment trains a semantic segmentation model to classify the image pixel level as SIM card or background, thereby avoiding complex preprocessing and parameter adjustment and improving the universality and practicality of the method.

[0112] Regarding the problem that manual intervention or subsequent manual correction is required to identify the extracted SIM card area: the existing methods usually require manual intervention or subsequent manual correction when identifying the extracted SIM card area, which increases the workload and cost. This embodiment can automatically correct the direction of the SIM card area through technologies such as connected domain analysis and minimum circumscribed rectangle, thereby reducing manual intervention and cost.

[0113] Furthermore, in the above step S1, in order to improve the generalization ability of the model, the collected images containing the SIM card can be enhanced by data enhancement, such as rotation, scaling, translation, cropping, adding noise, etc. This can increase the diversity of training samples, so that the model can maintain stable recognition performance when facing different SIM card images.

[0114] Furthermore, in the above step S2, a large-scale natural image dataset is used to pre-train the semantic segmentation model, and then fine-tuned on the SIM card image dataset. This can make full use of the common features learned by the pre-trained model, accelerate the convergence speed of the model, and may improve the final recognition accuracy.

[0115] Furthermore, in the above step S5, calculating the minimum bounding rectangle for each identified SIM card connectivity domain includes:

[0116] Direct calculation method: By analyzing the coordinate distribution of the two-dimensional image, the maximum and minimum values ​​of the coordinates are directly calculated to obtain the minimum enclosing rectangle.

[0117] Equal-interval rotation search method: The target two-dimensional image is rotated at equal intervals, and the parameters of the bounding rectangle of the contour in the coordinate system are recorded during each rotation process. The bounding rectangle with the smallest area is found by comparing these parameters. This method requires a large amount of calculation, but it can obtain more accurate results.

[0118] For SIM card images, since their shapes are relatively regular, the initial minimum bounding rectangle can be obtained directly using the direct calculation method. If higher accuracy is required, the equal-interval rotation search method can be used for further optimization.

[0119] After obtaining the minimum circumscribed rectangle, you need to calculate its rotation angle relative to the image coordinate system, which is achieved by comparing the angle between the long side of the rectangle and the horizontal axis of the image. The specific steps are as follows:

[0120] Determine the coordinates of the two diagonal points of the rectangle; calculate the slope of the straight line connecting the two diagonal points; and calculate the angle between the straight line and the horizontal axis of the image, that is, the required rotation angle, through the slope.

[0121] Correcting the direction of the SIM card area: Based on the calculated rotation angle, the direction of the SIM card area can be corrected using affine transformation or perspective transformation, including:

[0122] Affine transformation: Applicable to linear transformation and translation in a plane, which can maintain the linearity and parallelism of the image. For SIM card images, if their deformation is mainly rotation and translation, affine transformation can be used for correction.

[0123] Perspective transformation: Applicable to more complex deformation situations, such as projection distortion. If the SIM card image has obvious perspective effect or bending deformation, you may need to use perspective transformation to correct it. Perspective transformation requires the coordinates of 4 points to calculate the transformation matrix, so in practical applications you may need to manually select or automatically detect these feature points.

[0124] However, no matter which transformation method is used, it is necessary to adjust the corresponding parameters in the transformation matrix according to the calculated rotation angle, and then apply the transformation matrix to the SIM card area to achieve direction correction. The corrected SIM card image will be more suitable for subsequent OCR recognition processing.

[0125] Specifically, the method realizes the automatic recognition and extraction of SIM cards of arbitrary shapes, angles and partial occlusions by combining semantic segmentation, connected domain analysis, minimum bounding rectangle calculation, image transformation (affine transformation or perspective transformation) and OCR technology. The semantic segmentation model is one of the cores of this method, which can classify each pixel in the image as a SIM card or background. This means that even if the SIM card appears in any shape, angle, or is partially occluded, the semantic segmentation model can accurately identify the approximate area of ​​the SIM card based on the pixel-level classification information. A large number of image data sets containing SIM cards of various forms are used in the training process of the model, which enables the model to learn the characteristics of the SIM card, so that it can accurately identify the SIM card when faced with a new image.

[0126] On the probability map generated by the semantic segmentation model, after the binary map is obtained through threshold processing, all connected areas can be marked and extracted using the connected domain analysis technology. Among these connected areas, the SIM card area is included. The connected domain analysis technology can process connected areas of any shape, so even if the shape of the SIM card is irregular, it can be accurately extracted.

[0127] For each identified SIM card connected domain, calculate its minimum bounding rectangle. This rectangle is the smallest rectangle that encloses the SIM card area, and its angle reflects the direction of the SIM card. According to the angle of the minimum bounding rectangle, calculate the required rotation angle, and use affine transformation or perspective transformation to correct the direction of the SIM card area. In this way, no matter what angle the SIM card appears in the image, it can be corrected to the horizontal or vertical direction, which is convenient for subsequent OCR recognition.

[0128] The SIM card image after orientation correction is input into the OCR model for character segmentation and recognition. The OCR model can recognize the sequence of numbers and letters in the image, thereby realizing automatic recognition of the SIM card sequence. A large number of image datasets containing SIM card number / letter sequences are also used in the training process of the OCR model, so it can accurately recognize the characters on the SIM card.

[0129] Finally, the recognition results of the OCR model are converted into a structured data format (such as JSON or XML) for output. In this way, the recognition results can be conveniently used for subsequent management and application. This method realizes the automatic recognition and extraction of SIM cards of arbitrary shapes, angles and partial occlusions through the comprehensive application of semantic segmentation, connected domain analysis, minimum enclosing rectangle calculation, image transformation and OCR technology. This method not only improves the recognition accuracy, but also enhances the robustness and applicability of the system.

[0130] In summary, the present invention discloses a SIM card automatic recognition method based on semantic segmentation and shape analysis, which aims to use semantic segmentation technology to divide an image into two parts, a SIM card and a background, and then extract the SIM card area in combination with shape analysis. Specifically, a semantic segmentation model is first trained to classify the image pixel level into a SIM card or a background. Then the SIM card area is found by connected domain analysis, and its direction is corrected using techniques such as the minimum circumscribed rectangle. Finally, the corrected SIM card image is input into the OCR model to recognize the sequence of numbers and letters. The method can adaptively process SIM cards of arbitrary shapes, angles and partial occlusions, and is expected to achieve 97% SIM card extraction accuracy and 92% SIM card number recognition accuracy. The technology has important application value in the application scenarios of SIM card refined management, such as SIM card number batch inventory, inventory management, etc. Through the automated recognition process, the technical solution is expected to greatly improve the work efficiency of SIM card management, reduce errors in manual intervention, and provide strong technical support for the life cycle management of SIM cards.

[0131] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A SIM card automatic recognition method based on semantic segmentation and shape analysis, characterized in that: The following steps are involved: Collect image information including SIM card; Input the collected image information into a pre-trained semantic segmentation model, which classifies the image pixel level into SIM card or background and generates a probability map; wherein the probability map represents the probability of each pixel belonging to the SIM card or the background; Performing threshold processing on the probability map to generate a binary image, wherein the pixel value of the SIM card area is 1 and the pixel value of the background area is 0; Using connected domain analysis technology to mark and extract connected areas in the binary image, and identify the SIM card area; Calculate the minimum bounding rectangle for each identified SIM card connected domain, calculate the required rotation angle according to the angle of the minimum bounding rectangle, and use affine transformation or perspective transformation to correct the direction of the SIM card area to ensure that it is horizontal or vertical; Input the corrected SIM card image into the OCR model for character segmentation and recognition; The recognition result of the OCR model is converted into a structured data format and outputted.

2. The method according to claim 1, characterized in that: Use the camera to capture image information containing the SIM card, support image capture of a single SIM card and multiple randomly placed loose cards, and set the image resolution to 1080p; The image acquisition process is represented by the function I = f capture (resolution), resolution is the image resolution parameter, which is used to specify the resolution of the captured image.

3. The method according to claim 1, characterized in that The step of inputting the collected image information into a pre-trained semantic segmentation model includes: Use the pre-trained semantic segmentation model DeepLabV3+ to process the image; The DeepLabV3+ model uses dilated convolution and spatial pyramid pooling ASPP technology to extract multi-scale features in images; The model training strategy of the DeepLabV3+ model uses transfer learning to apply the pre-trained model weights to a SIM card-specific image dataset to accelerate the training process and improve the accuracy and efficiency of the model in SIM card recognition. During model training, a large number of image datasets containing SIM cards of various shapes, angles, and partial occlusions are used to improve the generalization and adaptability of the model; Among them, the semantic segmentation process is expressed as S = f seg (I,θ), where S is the segmented image, θ is the parameter of the model, and f seg is a semantic segmentation function, where the input image I is processed by the model parameters θ.

4. The method according to claim 3, characterized in that The construction of the semantic segmentation model DeepLabV3+ specifically includes: Building the model architecture: Using a pre-trained convolutional neural network as the backbone network to extract high-level features of the image, and combining it with the ASPP module with dilated convolutions with different dilation rates to capture multi-scale contextual information; Implement dilated convolution: insert holes between the elements of the convolution kernel to expand the receptive field and capture a wider range of contextual information in the image while keeping the computational complexity unchanged; Applying spatial pyramid pooling: Integrate a global average pooling layer in the ASPP module to extract global features of the image and fuse them with multi-scale feature maps to enhance the model’s understanding of the overall structure and context of the image; Add decoder module: By introducing a decoder module, the output of the ASPP module is upsampled to the same resolution as the input image using a deconvolution or upsampling layer to refine the segmentation result; Execute a transfer learning training strategy: initialize the model using pre-trained model weights and fine-tune it on a SIM card-specific image dataset so that the model learns the mapping relationship that classifies the image pixel level as SIM card or background; Generate output: The model outputs a probability map where the value of each pixel represents the probability of the pixel belonging to the SIM card or the background.

5. The method according to claim 1, characterized in that: Before the connected domain analysis, the probability map output by the semantic segmentation model is thresholded, including: Determine a threshold T according to the SIM card image characteristics, and this threshold is used to classify the pixel points in the probability map as SIM card or background; Compare each pixel value in the probability map with the determined threshold T one by one. If the pixel value is greater than or equal to T, the pixel is classified as a SIM card and marked as 1; if the pixel value is less than T, the pixel is classified as background and marked as 0; Based on the comparison results, a new binary image is generated, in which the pixels marked as 1 constitute the SIM card area, and the pixels marked as 0 constitute the background area.

6. The method according to claim 1, characterized in that: Connected domain analysis is expressed as the following formula: C=f cca (S,T) Among them, C is the connected domain set, T is the threshold parameter, and f cca is the connected domain analysis function.

7. The method according to any one of claims 1 to 6, characterized in that: The correction of the minimum bounding rectangle is expressed as the following formula: R=f rect (C,a) Among them, R is the corrected rectangular area, α is the rotation angle, and f rect is the minimum bounding rectangle correction function.

8. The method according to any one of claims 1 to 6, characterized in that: When performing character segmentation, projection segmentation or edge detection segmentation technology is used to accurately segment the continuous character sequence in the image into single characters; among them, projection segmentation determines the row and column boundaries of the characters by calculating the horizontal and vertical projections of the image, and edge detection segmentation uses edge detection algorithms to recognize the contours of characters to achieve segmentation.

9. The method according to any one of claims 1 to 6, characterized in that: When performing character recognition, a deep learning model combining convolutional neural network, long short-term memory network and connection temporal classification is used to recognize the segmented single character image; Use convolutional neural networks to extract features from segmented single character images; The extracted features are input into the long short-term memory network to capture the temporal dependencies in the character sequence and perform sequence modeling. The connection temporal classification is used as the loss function to train the model and handle the problem of mismatch between the input and output sequence lengths, and the recognized character sequence is directly output, thereby achieving accurate segmentation and recognition of the characters on the SIM card. The output character sequence is highly consistent with the actual character sequence on the SIM card.

10. The method according to any one of claims 1 to 6, characterized in that: Before inputting the collected image information into the semantic segmentation model, or before performing threshold processing on the probability map output by the semantic segmentation model, local feature extraction is first performed on the image or probability map; Based on the extracted local features, an adaptive threshold calculation algorithm is used to dynamically calculate the threshold corresponding to each area or pixel according to the local feature differences of different areas in the image or probability map; The calculated adaptive threshold is used to perform threshold processing on the image or probability map to generate a binary image. During the processing, each pixel or area is judged according to its corresponding adaptive threshold, and the pixel values ​​above the threshold are classified as the SIM card area, and the pixel values ​​below the threshold are classified as the background.

Images