Automatic ID card cutting and OCR recognition method and system based on AI

Abstract

The present invention provides an AI-based automatic ID card cutting and OCR recognition method and system, which relate to the field of image processing technology, including reconstructing a three-dimensional model of the document by using a three-dimensional point cloud registration network through multi-angle depth image and visible light image acquisition, and realizing image perspective correction in combination with attention posture estimation and projection matrix optimization; using a two-stream memory neural network to locate the information area, wherein a feature extraction backbone network constructs a multi-scale feature pyramid, and a dynamic memory update network performs feature matching and dictionary update based on a temporal attention module and contrast learning; performing character recognition on the segmented information sub-image through a multi-branch feature enhancement network, combining a self-attention enhancement unit and a context reasoning unit to improve the recognition accuracy, and using a reinforcement learning controller to dynamically optimize network parameters until a preset recognition credibility threshold is reached.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an AI-based automatic ID card cutting and OCR recognition method and system. Background Art

[0002] With the rapid development of artificial intelligence technology, the automatic recognition of ID card information has been widely used in many fields. Traditional ID card recognition methods mainly rely on plane image processing and OCR technology, and complete information extraction through preprocessing, segmentation and character recognition. However, in actual application scenarios, due to the influence of factors such as shooting angle, lighting conditions and reflection on the surface of the document, the acquired ID card images often have quality problems such as perspective deformation, blur distortion and partial occlusion, which seriously affect the accuracy and robustness of recognition.

[0003] There are still several deficiencies in the existing technology. The traditional two-dimensional image correction method is difficult to accurately handle perspective deformation in complex scenes, especially when the ID card is bent or wrinkled; the existing information area positioning algorithm has high requirements on image quality and is prone to positioning deviation when the lighting is uneven or there is reflection; the conventional OCR recognition method lacks an effective feature enhancement mechanism, has limited recognition ability for low-quality images, and is difficult to use historical recognition experience for continuous optimization; each processing link is relatively independent, and lacks an end-to-end optimization strategy, resulting in unstable overall recognition performance.

[0004] In summary, it is urgent to reconstruct the three-dimensional model of the certificate through multi-angle depth information to achieve accurate perspective correction; construct a two-stream neural network with memory learning ability to improve the accuracy and robustness of information area positioning; improve character recognition effect through adaptive feature enhancement and multi-model integrated recognition; use reinforcement learning to achieve dynamic optimization of recognition parameters to ensure continuous improvement of system performance. The present invention can solve the problems in the prior art. Summary of the invention

[0005] The embodiment of the present invention provides an AI-based automatic ID card cutting and OCR recognition method and system, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention,

[0007] Provided is an AI-based automatic ID card cutting and OCR recognition method, including:

[0008] Based on the depth camera array, multi-angle depth image data and visible light image data are collected, and the multi-angle depth image data are fused through the three-dimensional point cloud registration network to reconstruct the three-dimensional model of the ID card surface, extract the feature point sequence based on the three-dimensional model, calculate the spatial posture parameters through attention posture estimation, and use the projection matrix optimization network to establish a nonlinear correction model, perform perspective correction on the visible light image data, and obtain standardized image data;

[0009] The standardized image data is input into a dual-stream memory neural network, and the dual-stream memory neural network includes a feature extraction backbone network and a dynamic memory update network; the feature extraction backbone network extracts a local detail feature map and a global semantic feature map through a multi-scale feature pyramid structure; the dynamic memory update network performs feature matching fusion based on the historical region segmentation feature dictionary stored in the temporal attention module, combines the local detail feature map and the global semantic feature map, obtains the information region positioning result, and based on the information region positioning result, the standardized image data is segmented to obtain multiple information sub-images, and the historical region segmentation feature dictionary is updated through comparative learning;

[0010] Multiple information sub-images are input into a multi-branch feature enhancement network, and a self-attention enhancement unit of the multi-branch feature enhancement network first extracts a feature distribution map of the character area, and performs adaptive contrast enhancement based on the feature distribution map to obtain a feature enhanced image; the feature enhanced image is subjected to multi-model integrated recognition via a character association graph network constructed by a context reasoning unit to obtain an initial text recognition result; when the recognition credibility is lower than a preset threshold, the parameters of the self-attention enhancement unit are dynamically adjusted through a reinforcement learning controller to perform re-recognition until the credibility reaches the preset threshold, and a final recognition result is output.

[0011] In an optional embodiment,

[0012] Based on the depth camera array, multi-angle depth image data and visible light image data are collected, and the multi-angle depth image data are fused through the three-dimensional point cloud registration network to reconstruct the three-dimensional model of the ID card surface, extract the feature point sequence based on the three-dimensional model, calculate the spatial posture parameters through attention posture estimation, and use the projection matrix optimization network to establish a nonlinear correction model, perform perspective correction on the visible light image data, and obtain standardized image data including:

[0013] Based on a depth camera array, multi-angle depth image data and visible light image data are collected. The depth camera array consists of multiple depth cameras arranged in a ring at a preset elevation angle and a preset interval angle. Each of the depth cameras uses an adaptive exposure control algorithm to dynamically adjust exposure parameters according to scene light intensity, and realizes time alignment of multi-camera images based on a synchronous trigger signal between cameras;

[0014] The multi-angle depth image data is input into a three-dimensional point cloud registration network. The three-dimensional point cloud registration network extracts multi-scale features through layered sampling and grouped convolution to obtain a point cloud local area descriptor. Based on a cross-attention mechanism, a similarity matrix of the point cloud local area descriptor is calculated to perform feature matching. A random consistency sampling algorithm is combined to screen matching pairs, and a camera transformation matrix is ​​determined. Registration is performed through an iterative closest point algorithm, and the multi-angle depth image data is fused to reconstruct a three-dimensional model of the ID card surface.

[0015] Extracting three-dimensional feature points on the three-dimensional model of the ID card surface and constructing corresponding three-dimensional feature descriptors, extracting the spatial position relationship of the three-dimensional feature points through a spatial transformation network, weighting the geometric distribution, texture features and depth information of the three-dimensional feature points through an attention posture estimation network based on the spatial position relationship to obtain corresponding weighted feature descriptors, establishing a corresponding relationship between the three-dimensional model of the ID card surface and a standard template based on the weighted feature descriptors, and calculating spatial posture parameters using a perspective iteration algorithm;

[0016] The spatial posture parameters are input into the projection matrix optimization network, the perspective transformation parameters are predicted through the residual structure, pixel-level spatial mapping is performed in combination with differentiable bilinear interpolation, a regularization constraint based on surface features is introduced to establish a nonlinear correction model, image detail information is retained through jump connections, the perspective transformation parameters are optimized using adversarial training, and the visible light image data is perspective corrected to obtain standardized image data.

[0017] In an optional embodiment,

[0018] Extracting three-dimensional feature points on the three-dimensional model of the ID card surface and constructing corresponding three-dimensional feature descriptors, extracting the spatial position relationship of the three-dimensional feature points through a spatial transformation network, weighting the geometric distribution, texture features and depth information of the three-dimensional feature points through an attention posture estimation network based on the spatial position relationship, and obtaining corresponding weighted feature descriptors, establishing a corresponding relationship between the three-dimensional model of the ID card surface and a standard template based on the weighted feature descriptors, and using a perspective iteration algorithm to calculate spatial posture parameters includes:

[0019] Performing grid sampling on the three-dimensional model of the ID card surface to obtain a sampling point set, calculating the eigenvalue of the local neighborhood covariance matrix of each sampling point in the sampling point set to obtain the principal curvature, combining the principal curvature with the normal vector consistency constraint to construct a characteristic response function, and extracting three-dimensional feature points based on the characteristic response function;

[0020] Based on the local neighborhood information of each of the three-dimensional feature points, determining the radius of the feature support domain according to the local curvature change, constructing a local reference coordinate system in the feature support domain, dividing the feature support domain into a plurality of voxels, extracting the geometric features, depth information and texture features in each voxel, and combining the geometric features, the depth information and the texture features to obtain a three-dimensional feature descriptor;

[0021] Input the 3D feature descriptor into a spatial transformation network, extract the spatial position relationship of the 3D feature points by constructing a feature graph, wherein the feature graph uses the 3D feature points as vertices, uses the spatial connection relationship between the 3D feature points as edges, uses the 3D feature descriptors and position information as vertex features, and obtains spatial encoding features through graph attention convolution;

[0022] Input the spatial coding features into an attention posture estimation network, calculate the geometric distribution weights of the three-dimensional feature points through a spatial attention module, calculate the texture feature weights and depth information weights of the three-dimensional feature points through a channel attention module, and fuse the geometric distribution weights, the texture feature weights and the depth information weights with the spatial coding features to obtain a weighted feature descriptor;

[0023] The weighted feature descriptor and the standard template feature are subjected to Euclidean distance calculation to obtain a similarity matrix, a feature point matching scheme is determined, a one-to-one correspondence between the weighted feature descriptor and the standard template feature is established based on the feature point matching scheme, a corresponding point pair set is obtained, and a perspective iterative algorithm is used to calculate spatial posture parameters based on the corresponding point pair set.

[0024] In an optional embodiment,

[0025] The weighted feature descriptor and the standard template feature are subjected to Euclidean distance calculation to obtain a similarity matrix, a feature point matching scheme is determined, and a one-to-one correspondence between the weighted feature descriptor and the standard template feature is established based on the feature point matching scheme, and a corresponding point pair set is obtained, including:

[0026] Performing Euclidean distance calculation on the weighted feature descriptor and the standard template feature to obtain a feature similarity matrix, and converting the feature similarity matrix into a feature matching matrix, wherein the rows of the feature matching matrix correspond to the weighted feature descriptors, and the columns of the feature matching matrix correspond to the standard template features;

[0027] Subtract the minimum value of the corresponding row from each element in the feature matching matrix to obtain a first round update matrix, and subtract the minimum value of the corresponding column from each element in the first round update matrix to obtain a feature labeling matrix;

[0028] By marking the zero elements in the feature marker matrix, the minimum number of rows and the minimum number of columns are selected to cover all zero elements. If the minimum number of rows and the minimum number of columns are equal to the order of the feature marker matrix, a feature point matching solution is obtained; if the minimum number of rows and the minimum number of columns are less than the order of the feature marker matrix, the minimum value is found in the uncovered elements, an adjustment amount is determined, and the adjustment amount is subtracted from all uncovered elements, and the process is repeated until a feature point matching solution is obtained;

[0029] A one-to-one correspondence between the weighted feature descriptor and the standard template feature is established based on the feature point matching scheme to generate a corresponding point pair set.

[0030] In an optional embodiment,

[0031] The feature extraction backbone network extracts local detail feature maps and global semantic feature maps through a multi-scale feature pyramid structure, including:

[0032] Based on the preset resolution attenuation coefficient and channel expansion coefficient, in the order of gradually decreasing resolution and gradually increasing number of channels, the standardized image data is subjected to a feature extraction backbone network to extract a multi-level feature map;

[0033] The multi-level feature graphs are divided into a plurality of channel groups, and self-attention calculation is performed on the features in each channel group to obtain an intra-group attention feature. Based on a deformable convolutional network, a local spatial dependency relationship of the intra-group attention feature is calculated to obtain a spatial dependency weight, and the spatial dependency weight is weighted with the intra-group attention feature to obtain a first reconstructed feature;

[0034] Calculate the mutual information between feature maps of adjacent levels to obtain a feature correlation coefficient, perform down-sampling and transmission on the first reconstructed feature based on the feature correlation coefficient to obtain a down-sampled feature, perform up-sampling and transmission on the first reconstructed feature to obtain an up-sampled feature, wherein the transmission weight decays exponentially with increasing transmission distance, and combine the down-sampled feature with the up-sampled feature to obtain a second reconstructed feature;

[0035] Performing multi-head attention operations on the second reconstructed feature in the channel dimension and the spatial dimension respectively to obtain a channel dimension attention weight and a spatial dimension attention weight, weighting the channel dimension attention weight and the channel information of the second reconstructed feature to obtain a channel weighted feature, and weighting the spatial dimension attention weight and the spatial information of the second reconstructed feature to obtain a spatial weighted feature;

[0036] The channel weighted features and the spatial weighted features are weightedly combined to obtain a feature fusion map, feature consistency constraints are imposed between adjacent features in the feature fusion map to obtain a local detail feature map, feature response strengths are calculated for the feature fusion map, the feature response strengths are screened based on a preset feature response threshold to obtain a high response area, and regional aggregation constraints are imposed on the high response area to obtain a global semantic feature map.

[0037] In an optional embodiment,

[0038] The dynamic memory update network performs feature matching fusion based on the historical region segmentation feature dictionary stored in the temporal attention module, combined with the local detail feature map and the global semantic feature map, and obtains the information region positioning result including:

[0039] Performing online clustering of historical region segmentation features according to semantic similarity to obtain multiple feature categories, and storing the multiple feature categories in a historical region segmentation feature dictionary through a temporal attention module, wherein each feature category includes a category feature prototype and an intra-category feature variance, and the category feature prototype is weighted based on a dynamic update rate and an input historical region segmentation feature to obtain a first feature representation, and the dynamic update rate is calculated based on the intra-category feature variance, and a new feature category is created in the historical region segmentation feature dictionary when the distance between the input historical region segmentation feature and the category feature prototype is greater than a preset threshold;

[0040] Performing a non-local attention operation on the local detail feature map of the current frame to obtain a long-range dependency feature map, performing a channel attention operation on the global semantic feature map of the current frame to obtain a channel weighted feature map, performing feature matching on the long-range dependency feature map and the channel weighted feature map with the category feature prototype to obtain a first matching score and a second matching score, and combining them to obtain a feature matching score;

[0041] Calculating a first feature fusion weight and a second feature fusion weight based on the feature matching score, the intra-class feature variance and a temporal distance decay factor, wherein the temporal distance decay factor is inversely proportional to the time interval;

[0042] Performing a weighted combination of the first feature fusion weight and the long-range dependency feature map to obtain a first weighted feature map, performing a weighted combination of the second feature fusion weight and the channel weighted feature map to obtain a second weighted feature map, and combining the first weighted feature map and the second weighted feature map to obtain a feature fusion map;

[0043] Applying a spatiotemporal smoothness constraint to the feature fusion map to obtain a target feature map, wherein the spatiotemporal smoothness constraint includes a feature difference constraint item of adjacent time series frames and a feature gradient constraint item;

[0044] Based on the target feature map, region positioning is performed to obtain an information region positioning result.

[0045] In an optional embodiment,

[0046] A plurality of information sub-images are input into a multi-branch feature enhancement network, wherein a self-attention enhancement unit of the multi-branch feature enhancement network first extracts a feature distribution map of a character region, and performs adaptive contrast enhancement based on the feature distribution map to obtain a feature enhanced image; the feature enhanced image is subjected to multi-model integrated recognition via a character association graph network constructed by a contextual reasoning unit to obtain an initial text recognition result; when the recognition credibility is lower than a preset threshold, the parameters of the self-attention enhancement unit are dynamically adjusted by a reinforcement learning controller for re-recognition until the credibility reaches the preset threshold, and the output of the final recognition result includes:

[0047] Input multiple information sub-images into a multi-branch feature enhancement network, divide the multiple information sub-images into multiple regular grid units, perform multi-scale morphological analysis on the regular grid units to obtain local structural features, perform wavelet transform on the local structural features to obtain feature maps of different frequency components, perform adaptive weighted fusion on the feature maps of different frequency components, and construct a character region feature distribution map; calculate the feature difference between each region and the corresponding adjacent region in the character region feature distribution map to obtain a regional enhancement coefficient, perform adaptive contrast enhancement on the multiple information sub-images at different frequency components based on the regional enhancement coefficient, and reconstruct a feature enhanced image through inverse wavelet transform;

[0048] The feature-enhanced image is input into a context reasoning unit, the context reasoning unit constructs a character association graph network based on the spatial position relationship of the feature-enhanced image, extracts structural association features and semantic association features between characters from the character association graph network, and performs Fourier transform respectively to obtain frequency domain features, calculates feature phase consistency based on the frequency domain features to obtain a cross-scale feature matching graph, and performs multi-level feature fusion on the cross-scale feature matching graph with the structural association features and the semantic association features to obtain fused features;

[0049] The fusion features are input into a pre-trained text recognition model group to perform multi-model integrated recognition. In the multi-model integrated recognition process, a character transition probability matrix is ​​constructed based on a conditional random field. The results of the multi-model integrated recognition are sequence-corrected according to the transition probability matrix to obtain multiple groups of character sequence recognition results and corresponding recognition credibility. The multiple groups of character sequence recognition results are probability-weighted using a softmax function with an adaptive temperature coefficient to obtain an initial text recognition result and a corresponding comprehensive credibility.

[0050] When the comprehensive credibility is lower than a preset credibility threshold, a reward function is constructed based on the comprehensive credibility through a reinforcement learning controller, a regularization constraint term based on prior knowledge of character structure is introduced into the reward function, and a strategy iteration algorithm with momentum term is adopted to calculate the adjustment direction of feature enhancement parameters, and the network parameters in the multi-branch feature enhancement network are simultaneously updated according to the adjustment direction to regenerate a new feature enhanced image; the iteration is repeated until the comprehensive credibility is higher than the preset credibility threshold, and the final recognition result is output.

[0051] According to a second aspect of the embodiments of the present invention,

[0052] Provide an AI-based automatic ID card cutting and OCR recognition system, including:

[0053] The first unit is used to collect multi-angle depth image data and visible light image data based on a depth camera array, fuse the multi-angle depth image data through a three-dimensional point cloud registration network, reconstruct a three-dimensional model of the ID card surface, extract a feature point sequence based on the three-dimensional model, calculate spatial posture parameters through attention posture estimation, establish a nonlinear correction model using a projection matrix optimization network, perform perspective correction on the visible light image data, and obtain standardized image data;

[0054] The second unit is used to input the standardized image data into a dual-stream memory neural network, and the dual-stream memory neural network includes a feature extraction backbone network and a dynamic memory update network; the feature extraction backbone network extracts a local detail feature map and a global semantic feature map through a multi-scale feature pyramid structure; the dynamic memory update network performs feature matching fusion based on the historical region segmentation feature dictionary stored in the temporal attention module, combines the local detail feature map and the global semantic feature map, obtains the information region positioning result, and based on the information region positioning result, divides the standardized image data to obtain multiple information sub-images, and updates the historical region segmentation feature dictionary through comparative learning;

[0055] The third unit is used to input multiple information sub-images into a multi-branch feature enhancement network, wherein the self-attention enhancement unit of the multi-branch feature enhancement network first extracts the feature distribution map of the character area, and performs adaptive contrast enhancement based on the feature distribution map to obtain a feature enhanced image; the feature enhanced image is subjected to multi-model integrated recognition via a character association graph network constructed by a context reasoning unit to obtain an initial text recognition result; when the recognition credibility is lower than a preset threshold, the parameters of the self-attention enhancement unit are dynamically adjusted through a reinforcement learning controller to re-recognize until the credibility reaches the preset threshold, and a final recognition result is output.

[0056] According to a third aspect of the embodiments of the present invention,

[0057] An electronic device is provided, comprising:

[0058] processor;

[0059] a memory for storing processor-executable instructions;

[0060] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0061] A fourth aspect of the embodiments of the present invention is:

[0062] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0063] In the embodiment of the present invention, three-dimensional information is acquired through a depth camera array, and perspective correction is performed in combination with attention posture estimation, which effectively solves the problem of image deformation caused by factors such as angle and lighting during the ID card image acquisition process, thereby improving the accuracy of subsequent OCR recognition; a dual-stream memory neural network is adopted, and a dynamic memory update network is used to store historical region segmentation features, and matching and fusion are performed in combination with multi-scale features, which can more accurately locate each information area on the ID card, and maintain a high positioning accuracy even when there are differences in the layout of the ID card information; a multi-branch feature enhancement network is used to enhance character features and context reasoning, and combined with a reinforcement learning controller to dynamically adjust recognition parameters, which can effectively improve the accuracy and credibility of OCR recognition. At the same time, the historical region segmentation feature dictionary is continuously updated through comparative learning, which can further improve the robustness and adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 The figure is a flow chart of an automatic ID card cutting and OCR recognition method based on AI according to an embodiment of the present invention;

[0065] Figure 2 The figure is a schematic diagram of the structure of an automatic ID card cutting and OCR recognition system based on AI according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0068] Figure 1 FIG. 1 is a flow chart of an automatic ID card cutting and OCR recognition method based on AI according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0069] S101. Based on the depth camera array, multi-angle depth image data and visible light image data are collected, and the multi-angle depth image data are fused through the three-dimensional point cloud registration network to reconstruct the three-dimensional model of the ID card surface, extract the feature point sequence based on the three-dimensional model, calculate the spatial posture parameters through attention posture estimation, and use the projection matrix optimization network to establish a nonlinear correction model, perform perspective correction on the visible light image data, and obtain standardized image data;

[0070] In an optional specific implementation, multi-angle image data of the ID card is collected based on a depth camera array, the depth camera array includes at least three depth cameras and one visible light camera, the at least three depth cameras are arranged in a circular array, the visible light camera is located at the center of the circular array, the at least three depth cameras collect depth image data respectively, and the visible light camera collects visible light image data;

[0071] The multi-angle depth image data is input into a three-dimensional point cloud registration network, the three-dimensional point cloud registration network calculates the three-dimensional point cloud registration parameters between adjacent perspectives through an iterative closest point algorithm, and the depth image data of different perspectives are fused based on the three-dimensional point cloud registration parameters to reconstruct a three-dimensional model of the ID card surface; geometric features such as corner points and edges are extracted from the three-dimensional model to construct a feature point sequence;

[0072] The feature point sequence is input into an attention posture estimation module, and the attention posture estimation module calculates the rotation matrix and translation vector of the ID card based on the spatial distribution of the feature point sequence to obtain spatial posture parameters; and constructs the initial value of the projection transformation matrix according to the spatial posture parameters;

[0073] Inputting the initial value of the projection transformation matrix and the visible light image data into a projection matrix optimization network, the projection matrix optimization network constructs a loss function based on the similarity of image region features, iteratively optimizes the parameters of the projection transformation matrix through gradient descent, and establishes a nonlinear correction model that takes into account lens distortion;

[0074] The perspective correction and distortion correction are performed on the visible light image data based on the nonlinear correction model to obtain standardized image data.

[0075] In this embodiment, multi-angle depth image data are fused through a three-dimensional point cloud registration network to reconstruct the three-dimensional model of the ID card surface and achieve highly accurate structural reproduction; a feature point sequence is extracted based on the three-dimensional model, and the spatial posture parameters are calculated through the attention posture estimation module to improve the accuracy of posture calculation; a nonlinear correction model is established using the projection matrix optimization network to effectively correct the perspective distortion of visible light images; standardized image data is obtained after correction to ensure the consistency and quality of the image, providing reliable input for subsequent processing; the combination of depth images and visible light data has the ability to adapt to complex lighting and angle conditions, thereby enhancing the robustness of the method.

[0076] S102. Input the standardized image data into a dual-stream memory neural network, the dual-stream memory neural network includes a feature extraction backbone network and a dynamic memory update network; the feature extraction backbone network extracts a local detail feature map and a global semantic feature map through a multi-scale feature pyramid structure; the dynamic memory update network performs feature matching and fusion based on the historical region segmentation feature dictionary stored in the temporal attention module, combined with the local detail feature map and the global semantic feature map, to obtain an information region positioning result, and based on the information region positioning result, the standardized image data is segmented to obtain a plurality of information sub-images, and the historical region segmentation feature dictionary is updated through comparative learning;

[0077] In an optional specific implementation, the standardized image data is input into a feature extraction backbone network, wherein the feature extraction backbone network includes five convolutional layers, each of which is followed by a batch normalization layer and a ReLU activation function; a feature pyramid network is constructed between the output feature maps of the third, fourth, and fifth convolutional layers, and the fusion of features of different scales is achieved through upsampling and downsampling operations to extract local detail feature maps; the output feature map of the fifth convolutional layer is subjected to global average pooling and a fully connected layer to obtain a global semantic feature map;

[0078] Constructing a dynamic memory update network, the dynamic memory update network comprising a temporal attention module and a feature dictionary update module; the temporal attention module maintains a historical region segmentation feature dictionary of a preset capacity, each feature dictionary item comprising a feature vector and a corresponding confidence weight;

[0079] Calculate the similarity scores between the local detail feature map and the global semantic feature map and each feature vector in the historical region segmentation feature dictionary, construct an attention map based on the similarity scores and confidence weights, perform weighted fusion of the attention map with the local detail feature map and the global semantic feature map, and output a probability map of each pixel belonging to the information region through two fully connected layers;

[0080] Performing threshold segmentation and connected domain analysis on the probability map, extracting the minimum bounding rectangle of each information area as a positioning result, and segmenting the standardized image data based on the positioning result to obtain multiple information sub-images;

[0081] A contrastive learning method is used to update the historical region segmentation feature dictionary, specifically including: taking the features extracted from the current batch as positive samples, randomly selecting features in the historical feature dictionary as negative samples, and constructing a contrastive loss function; based on the gradient of the contrastive loss function, updating the feature vectors and confidence weights in the feature dictionary, and maintaining the dictionary capacity according to the first-in-first-out principle.

[0082] In this embodiment, the multi-scale feature pyramid structure of the feature extraction backbone network is used to accurately capture local details and global semantic information, thereby improving the comprehensiveness and accuracy of information extraction; the temporal attention module of the dynamic memory update network is combined with the historical region segmentation feature dictionary to enhance the timeliness and accuracy of feature matching; through feature matching and fusion, the accuracy of information region positioning is effectively improved, laying the foundation for subsequent processing; image segmentation based on the positioning results can quickly obtain multiple high-quality information sub-images, thereby improving processing efficiency; by updating the historical feature dictionary through comparative learning, the system can continuously optimize the segmentation performance and enhance its adaptability to dynamically changing scenes.

[0083] S013. Input multiple information sub-images into a multi-branch feature enhancement network, wherein the self-attention enhancement unit of the multi-branch feature enhancement network first extracts the feature distribution map of the character area, and performs adaptive contrast enhancement based on the feature distribution map to obtain a feature enhanced image; the feature enhanced image is subjected to multi-model integrated recognition via a character association graph network constructed by a context reasoning unit to obtain an initial text recognition result; when the recognition credibility is lower than a preset threshold, the parameters of the self-attention enhancement unit are dynamically adjusted through a reinforcement learning controller to re-recognize until the credibility reaches the preset threshold, and the final recognition result is output.

[0084] In this embodiment, the character area feature distribution map is extracted by the self-attention enhancement unit to achieve targeted contrast adjustment and improve the recognizability of character details in the image; the character association graph network is constructed by using the context reasoning unit to effectively capture the semantic relationship between characters and improve the accuracy of the recognition results; the multi-model integration strategy is combined to improve the robustness and comprehensive performance of text recognition; the parameters of the self-attention enhancement unit are dynamically adjusted by the reinforcement learning controller to continuously optimize the recognition process and ensure the credibility of the recognition results; the results are output only after the recognition credibility reaches the preset threshold to ensure the high accuracy and reliability of the final recognition.

[0085] In an optional implementation, based on a depth camera array, multi-angle depth image data and visible light image data are collected, and the multi-angle depth image data are fused through a three-dimensional point cloud registration network to reconstruct a three-dimensional model of the ID card surface, extract a feature point sequence based on the three-dimensional model, calculate the spatial posture parameters through attention posture estimation, and use a projection matrix optimization network to establish a nonlinear correction model, perform perspective correction on the visible light image data, and obtain standardized image data including:

[0086] Based on a depth camera array, multi-angle depth image data and visible light image data are collected. The depth camera array consists of multiple depth cameras arranged in a ring at a preset elevation angle and a preset interval angle. Each of the depth cameras uses an adaptive exposure control algorithm to dynamically adjust exposure parameters according to scene light intensity, and realizes time alignment of multi-camera images based on a synchronous trigger signal between cameras;

[0087] The multi-angle depth image data is input into a three-dimensional point cloud registration network. The three-dimensional point cloud registration network extracts multi-scale features through layered sampling and grouped convolution to obtain a point cloud local area descriptor. Based on a cross-attention mechanism, a similarity matrix of the point cloud local area descriptor is calculated to perform feature matching. A random consistency sampling algorithm is combined to screen matching pairs, and a camera transformation matrix is ​​determined. Registration is performed through an iterative closest point algorithm, and the multi-angle depth image data is fused to reconstruct a three-dimensional model of the ID card surface.

[0088] Extracting three-dimensional feature points on the three-dimensional model of the ID card surface and constructing corresponding three-dimensional feature descriptors, extracting the spatial position relationship of the three-dimensional feature points through a spatial transformation network, weighting the geometric distribution, texture features and depth information of the three-dimensional feature points through an attention posture estimation network based on the spatial position relationship to obtain corresponding weighted feature descriptors, establishing a corresponding relationship between the three-dimensional model of the ID card surface and a standard template based on the weighted feature descriptors, and calculating spatial posture parameters using a perspective iteration algorithm;

[0089] The spatial posture parameters are input into the projection matrix optimization network, the perspective transformation parameters are predicted through the residual structure, pixel-level spatial mapping is performed in combination with differentiable bilinear interpolation, a regularization constraint based on surface features is introduced to establish a nonlinear correction model, image detail information is retained through jump connections, the perspective transformation parameters are optimized using adversarial training, and the visible light image data is perspective corrected to obtain standardized image data.

[0090] In an optional specific implementation, a circular array consisting of 8 depth cameras is built, with each depth camera spaced 45 degrees apart and the elevation angle set to 15 degrees. A rotatable platform is placed in the center of the array for placing the ID card. Each depth camera is equipped with an adaptive exposure control algorithm, which can dynamically adjust the exposure parameters according to the light intensity on the surface of the ID card to ensure that the collected image data is clear. All depth cameras are controlled by a synchronous trigger signal to ensure the timing alignment of multi-camera images and avoid image deviation caused by time difference. For example, when the ID card is placed on a rotating platform, the synchronous trigger signal will simultaneously trigger 8 depth cameras to collect images and obtain multi-angle depth image data and corresponding visible light image data.

[0091] The collected multi-angle depth image data is input into the 3D point cloud registration network. The network first performs hierarchical sampling on the point cloud data, and then uses grouped convolution to extract multi-scale features to obtain local area descriptors of the point cloud. For example, the point cloud is divided into multiple groups, each containing a number of points, and convolution operations are performed on each group of point clouds. After that, the network uses the cross-attention mechanism to calculate the similarity matrix between these descriptors for feature matching. For example, the similarity score between descriptor A and descriptor B is calculated. Next, the random consistency sampling algorithm is used to screen matching pairs, eliminate false matches, and determine the camera transformation matrix. Finally, the iterative closest point algorithm is used for registration, and the point cloud data from multiple angles are fused together to reconstruct a complete 3D model of the ID card surface. Assuming there are two point clouds P1 and P2, the iterative closest point algorithm will iterate continuously to find the nearest point of each point in P1 in P2, and calculate the transformation matrix, and finally register P1 and P2 together.

[0092] Feature points are extracted from the reconstructed three-dimensional model of the ID card surface, such as the corner points of the ID card number area, and the corresponding three-dimensional feature descriptors are constructed to describe the local geometric information of each feature point. Then, the spatial position relationship of the three-dimensional feature points is extracted using the spatial transformation network. For example, the relative position of feature point A relative to feature point B is calculated. Next, the attention posture estimation network weights the feature points according to the spatial position relationship, texture features, and depth information to obtain the corresponding weighted feature descriptors. For example, feature points with more important spatial position relationships are given higher weights. Finally, the weighted feature descriptors are matched with the feature descriptors of the standard template to establish the corresponding relationship between the three-dimensional model of the ID card surface and the standard template, and the perspective iteration algorithm is used to calculate the spatial posture parameters, such as the rotation angle and translation, to describe the posture of the ID card in three-dimensional space.

[0093] The calculated ID card spatial posture parameters are input into the projection matrix optimization network. The network predicts perspective transformation parameters through the residual structure. For example, the scaling, rotation, and translation parameters required for the ID card image are predicted. Then, differentiable bilinear interpolation is combined for pixel-level spatial mapping to project the ID card image onto the standard plane. In order to improve the correction accuracy, regularization constraints based on surface features are introduced to establish a nonlinear correction model. For example, constraints are imposed based on the curvature information of the ID card surface, and image detail information is retained through jump connections. The network uses adversarial training to optimize the perspective transformation parameters, so that the corrected image is closer to the standard template. Finally, the visible light image data is perspective corrected to obtain standardized image data.

[0094] In this embodiment, a depth camera array is used to synchronously collect multi-angle image data, which greatly improves the data collection efficiency compared with the traditional single camera sequential collection method; the three-dimensional point cloud registration network and the attention posture estimation network can more accurately reconstruct the three-dimensional model of the ID card surface and estimate the spatial posture parameters, thereby improving the accuracy of subsequent perspective correction; the adaptive exposure control algorithm and the regularization constraint based on surface features can effectively cope with complex situations such as different lighting conditions and ID card surface deformation, thereby enhancing the robustness of the system.

[0095] In an optional implementation, three-dimensional feature points are extracted from the three-dimensional model of the ID card surface, and corresponding three-dimensional feature descriptors are constructed. The spatial position relationship of the three-dimensional feature points is extracted through a spatial transformation network. Based on the spatial position relationship, the geometric distribution, texture features and depth information of the three-dimensional feature points are weighted through an attention posture estimation network to obtain a corresponding weighted feature descriptor. Based on the weighted feature descriptor, a corresponding relationship between the three-dimensional model of the ID card surface and a standard template is established. The spatial posture parameters are calculated using a perspective iteration algorithm, including:

[0096] Performing grid sampling on the three-dimensional model of the ID card surface to obtain a sampling point set, calculating the eigenvalue of the local neighborhood covariance matrix of each sampling point in the sampling point set to obtain the principal curvature, combining the principal curvature with the normal vector consistency constraint to construct a characteristic response function, and extracting three-dimensional feature points based on the characteristic response function;

[0097] Based on the local neighborhood information of each of the three-dimensional feature points, determining the radius of the feature support domain according to the local curvature change, constructing a local reference coordinate system in the feature support domain, dividing the feature support domain into a plurality of voxels, extracting the geometric features, depth information and texture features in each voxel, and combining the geometric features, the depth information and the texture features to obtain a three-dimensional feature descriptor;

[0098] Input the 3D feature descriptor into a spatial transformation network, extract the spatial position relationship of the 3D feature points by constructing a feature graph, wherein the feature graph uses the 3D feature points as vertices, uses the spatial connection relationship between the 3D feature points as edges, uses the 3D feature descriptors and position information as vertex features, and obtains spatial encoding features through graph attention convolution;

[0099] Input the spatial coding features into an attention posture estimation network, calculate the geometric distribution weights of the three-dimensional feature points through a spatial attention module, calculate the texture feature weights and depth information weights of the three-dimensional feature points through a channel attention module, and fuse the geometric distribution weights, the texture feature weights and the depth information weights with the spatial coding features to obtain a weighted feature descriptor;

[0100] The weighted feature descriptor and the standard template feature are subjected to Euclidean distance calculation to obtain a similarity matrix, a feature point matching scheme is determined, a one-to-one correspondence between the weighted feature descriptor and the standard template feature is established based on the feature point matching scheme, a corresponding point pair set is obtained, and a perspective iterative algorithm is used to calculate spatial posture parameters based on the corresponding point pair set.

[0101] In an optional specific implementation, the obtained three-dimensional model of the ID card surface is preprocessed, such as denoising, smoothing and other operations, to improve the accuracy and stability of subsequent processing. Then, the preprocessed three-dimensional model is grid sampled to obtain a uniformly distributed set of sampling points. To avoid the influence of boundary effects, methods such as Poisson disk sampling can be used to ensure that the sampling points are evenly distributed on the model surface. For example, the sampling point spacing is set to 0.5 mm, and about 10,000 sampling points are obtained on the ID card surface.

[0102] Next, the eigenvalues ​​of the local neighborhood covariance matrix of each sampling point are calculated to obtain the two principal curvatures of the point. The principal curvature reflects the degree of curvature of the surface in different directions at the point. At the same time, the normal vector of each sampling point is calculated to indicate the normal direction of the surface at the point. The principal curvature is combined with the normal vector consistency constraint to construct a feature response function. For example, the idea of ​​the Harris operator or SIFT operator can be adopted to integrate the principal curvature and normal vector information into the feature response function. Based on the feature response function, points with significant local features are extracted as three-dimensional feature points. For example, the top 500 sampling points with the feature response function values ​​are selected as three-dimensional feature points.

[0103] For each extracted 3D feature point, the radius of the feature support domain is adaptively determined according to the curvature change of its local neighborhood. The support domain radius is smaller in areas with large curvature changes, and larger in areas with small curvature changes. In the feature support domain, a local reference coordinate system is constructed, and the support domain is divided into multiple voxels. For example, the support domain is divided into a 5×5×5 voxel grid. The geometric features (e.g., the average curvature, Gaussian curvature, shape diameter function, etc. of the point cloud in the voxel), depth information (e.g., the distance from the voxel center to the feature point), and texture features (e.g., statistical features of the color or grayscale value in the voxel) in each voxel are extracted. These features are combined to form a feature descriptor of the 3D feature point. For example, the geometric features, depth information, and texture features are concatenated into a 128-dimensional vector as the 3D feature descriptor of the feature point.

[0104] The descriptors of all 3D feature points are input into the spatial transformer network. The spatial transformer network extracts the spatial position relationship between 3D feature points by constructing a feature graph. The feature graph uses 3D feature points as vertices and the spatial connection relationship between feature points as edges. The 3D feature descriptors and position information are used as vertex features, and the spatial encoding features are obtained through the graph attention convolution operation. The spatial encoding features combine the descriptor information of the feature points themselves and the spatial relationship information in the local neighborhood.

[0105] The spatial encoding features are input into the attention pose estimation network. The network contains a spatial attention module and a channel attention module. The spatial attention module focuses on the geometric distribution of feature points in three-dimensional space and assigns different weights to each feature point. The channel attention module focuses on the importance of different channels in the feature descriptor and assigns different weights to texture features and depth information respectively. These weights are fused with the spatial encoding features to obtain weighted feature descriptors.

[0106] The weighted feature descriptor is used to calculate the Euclidean distance with the pre-stored standard template features to obtain a similarity matrix. Based on the similarity matrix, a feature point matching scheme is determined. For example, a one-to-one correspondence between the weighted feature descriptor and the standard template features is established by using methods such as nearest neighbor matching or bidirectional matching to obtain a corresponding point pair set. Finally, a perspective iteration algorithm is used to calculate the posture parameters of the three-dimensional model of the ID card surface in space based on the corresponding point pair set, such as a rotation matrix and a translation vector.

[0107] In this embodiment, by combining local feature descriptors, spatial relationship information and attention mechanism, the method can more effectively capture the key features of the three-dimensional model of the ID card surface, thereby improving the accuracy and robustness of posture estimation; the use of local feature descriptors and spatial transformation networks can effectively cope with the influence of noise and partial occlusion, and even when there is a certain degree of contamination or occlusion on the surface of the ID card, its posture parameters can still be accurately estimated; by adaptively determining the feature support domain radius and adopting an efficient attention mechanism, the method can reduce the computational complexity and improve the computational efficiency while ensuring the accuracy of posture estimation, and is suitable for real-time application scenarios.

[0108] In an optional implementation, the weighted feature descriptor and the standard template feature are subjected to Euclidean distance calculation to obtain a similarity matrix, a feature point matching scheme is determined, and a one-to-one correspondence between the weighted feature descriptor and the standard template feature is established based on the feature point matching scheme, and the corresponding point pair set obtained includes:

[0109] Performing Euclidean distance calculation on the weighted feature descriptor and the standard template feature to obtain a feature similarity matrix, and converting the feature similarity matrix into a feature matching matrix, wherein the rows of the feature matching matrix correspond to the weighted feature descriptors, and the columns of the feature matching matrix correspond to the standard template features;

[0110] Subtract the minimum value of the corresponding row from each element in the feature matching matrix to obtain a first round update matrix, and subtract the minimum value of the corresponding column from each element in the first round update matrix to obtain a feature labeling matrix;

[0111] By marking the zero elements in the feature marker matrix, the minimum number of rows and the minimum number of columns are selected to cover all zero elements. If the minimum number of rows and the minimum number of columns are equal to the order of the feature marker matrix, a feature point matching solution is obtained; if the minimum number of rows and the minimum number of columns are less than the order of the feature marker matrix, the minimum value is found in the uncovered elements, an adjustment amount is determined, and the adjustment amount is subtracted from all uncovered elements, and the process is repeated until a feature point matching solution is obtained;

[0112] A one-to-one correspondence between the weighted feature descriptor and the standard template feature is established based on the feature point matching scheme to generate a corresponding point pair set.

[0113] In an optional specific implementation, first, feature points of the image to be matched and the template image are extracted, and weighted feature descriptors and standard template features of each feature point are calculated. The weighted feature descriptors can contain a variety of feature information, such as color, texture, gradient, etc., and are assigned different weights according to their importance. The standard template features are pre-extracted features used for matching.

[0114] Next, the Euclidean distance between each weighted feature descriptor and all standard template features is calculated. The smaller the Euclidean distance, the more similar the two features are. All distance values ​​are stored in a matrix called a feature similarity matrix. The rows of this matrix correspond to weighted feature descriptors and the columns correspond to standard template features. For example, assuming there are 3 weighted feature descriptors and 4 standard template features, the feature similarity matrix is ​​a matrix with 3 rows and 4 columns.

[0115] Then, convert the feature similarity matrix into a feature matching matrix. The specific method is: for each row, find the minimum value of the row, and subtract this minimum value from all elements in the row. Then do the same operation for each column, find the minimum value of the column, and subtract this minimum value from all elements in the column. For example, assuming that the minimum value of the first row is 0.5, then subtract 0.5 from all elements in the first row. Assuming that the minimum value of the first column is 0.2, then subtract 0.2 from all elements in the first column. The final matrix is ​​the feature matching matrix.

[0116] Next, mark the feature matching matrix to determine the feature point matching scheme. In the feature matching matrix, find all elements with zero values ​​and mark them. Then, try to cover all marked zero elements with the minimum number of rows and columns. If the minimum number of rows and columns is equal to the order of the matrix (that is, the number of rows or columns), it means that a perfect matching scheme has been found. For example, for a matrix with 3 rows and 4 columns, if 3 rows and 4 columns are required to cover all zero elements, then a matching scheme has been found. If the minimum number of rows and columns is less than the order of the matrix, it means that there are some uncovered zero elements. Find the minimum value among the uncovered elements and subtract this minimum value from all uncovered elements. Then repeat the above marking and covering process until a matching scheme is found. For example, if the minimum value of the uncovered elements is 0.1, subtract 0.1 from all uncovered elements.

[0117] Finally, according to the obtained feature point matching scheme, a one-to-one correspondence between the weighted feature descriptors and the standard template features is established to generate a corresponding point pair set. The corresponding point pair set contains matching feature point pairs, for example, the first weighted feature descriptor matches the third standard template feature, the second weighted feature descriptor matches the first standard template feature, and so on.

[0118] In this embodiment, through weighted feature descriptors and optimized matching schemes, matching feature points can be found more accurately, thereby improving matching accuracy; it has a certain degree of robustness to factors such as image noise and lighting changes, and can adapt to different application scenarios; compared with traditional matching methods, this method has lower computational complexity and can complete the matching process faster.

[0119] In an optional implementation, the feature extraction backbone network extracts local detail feature maps and global semantic feature maps through a multi-scale feature pyramid structure, including:

[0120] Based on the preset resolution attenuation coefficient and channel expansion coefficient, in the order of gradually decreasing resolution and gradually increasing number of channels, the standardized image data is subjected to a feature extraction backbone network to extract a multi-level feature map;

[0121] The multi-level feature graphs are divided into a plurality of channel groups, and self-attention calculation is performed on the features in each channel group to obtain an intra-group attention feature. Based on a deformable convolutional network, a local spatial dependency relationship of the intra-group attention feature is calculated to obtain a spatial dependency weight, and the spatial dependency weight is weighted with the intra-group attention feature to obtain a first reconstructed feature;

[0122] Calculate the mutual information between feature maps of adjacent levels to obtain a feature correlation coefficient, perform down-sampling and transmission on the first reconstructed feature based on the feature correlation coefficient to obtain a down-sampled feature, perform up-sampling and transmission on the first reconstructed feature to obtain an up-sampled feature, wherein the transmission weight decays exponentially with increasing transmission distance, and combine the down-sampled feature with the up-sampled feature to obtain a second reconstructed feature;

[0123] Performing multi-head attention operations on the second reconstructed feature in the channel dimension and the spatial dimension respectively to obtain a channel dimension attention weight and a spatial dimension attention weight, weighting the channel dimension attention weight and the channel information of the second reconstructed feature to obtain a channel weighted feature, and weighting the spatial dimension attention weight and the spatial information of the second reconstructed feature to obtain a spatial weighted feature;

[0124] The channel weighted features and the spatial weighted features are weightedly combined to obtain a feature fusion map, feature consistency constraints are imposed between adjacent features in the feature fusion map to obtain a local detail feature map, feature response strengths are calculated for the feature fusion map, the feature response strengths are screened based on a preset feature response threshold to obtain a high response area, and regional aggregation constraints are imposed on the high response area to obtain a global semantic feature map.

[0125] In an optional specific implementation, first, prepare the image data to be processed. In order to ensure the effectiveness of subsequent processing, the image data needs to be standardized, for example, the image pixel values ​​are normalized to between 0 and 1, or zero mean unit variance processing is performed. Taking an RGB image with a size of 224×224 as an example, after the standardization process, the pixel value range of the image data becomes between 0 and 1.

[0126] Then, a pre-trained feature extraction backbone network, such as ResNet, VGG, etc., is used to extract features from the standardized image data. The backbone network adopts a multi-scale feature pyramid structure and can extract feature maps at different levels. The design of the backbone network follows the principle of gradually decreasing resolution and gradually increasing the number of channels. Assuming that the preset resolution attenuation coefficient is 0.5 and the channel expansion coefficient is 2, for the input 224×224 image, the backbone network can extract multi-level feature maps such as 112×112×64, 56×56×128, 28×28×256, etc.

[0127] Next, the extracted multi-level feature maps are grouped and self-attention is calculated. The feature maps of each level are divided into multiple channel groups. For example, the feature maps of 256 channels are divided into 4 channel groups, each containing 64 channels. In each channel group, self-attention calculation is performed to obtain the intra-group attention features. Then, the local spatial dependency of the intra-group attention features is calculated using a deformable convolutional network to obtain the spatial dependency weights. The spatial dependency weights are weighted with the intra-group attention features to obtain the first reconstructed features.

[0128] Subsequently, the mutual information between feature maps of adjacent levels is calculated to obtain the feature correlation coefficient. For example, the mutual information between a 56×56×128 feature map and a 28×28×256 feature map is calculated. Based on the feature correlation coefficient, the first reconstructed feature is downsampled and upsampled. The downsampled transfer transfers the high-resolution feature to the low-resolution feature, and the upsampled transfer transfers the low-resolution feature to the high-resolution feature. The transfer weight decays exponentially with the increase of the transfer distance. The downsampled feature and the upsampled feature are combined to obtain the second reconstructed feature.

[0129] Next, multi-head attention operations are performed on the second reconstructed feature in the channel dimension and the spatial dimension respectively to obtain the channel dimension attention weight and the spatial dimension attention weight. The channel dimension attention weight is weighted with the channel information of the second reconstructed feature to obtain the channel weighted feature. The spatial dimension attention weight is weighted with the spatial information of the second reconstructed feature to obtain the spatial weighted feature.

[0130] Then, the channel weighted features are weightedly combined with the spatial weighted features to obtain a feature fusion map. Feature consistency constraints are imposed between adjacent features in the feature fusion map, for example, a smoothing filter is used to obtain a local detail feature map. The feature response strength is calculated for the feature fusion map, for example, the absolute value of each feature is calculated. Based on a preset feature response threshold, such as 0.5, the feature response strength is screened to obtain a high response area. Regional aggregation constraints are imposed on the high response area, for example, a maximum pooling operation is used to obtain a global semantic feature map.

[0131] In this embodiment, through multi-scale feature pyramid structure, self-attention mechanism, deformable convolution and feature transfer operations, local detail information and global semantic information in the image can be captured more effectively, thereby improving the expressiveness of the features; feature consistency constraints and regional aggregation constraints can effectively suppress the interference of noise and irrelevant information and enhance the robustness of the features; channel grouping and multi-head attention mechanism can reduce computational complexity and improve computational efficiency.

[0132] In an optional implementation, the dynamic memory update network performs feature matching fusion based on the historical region segmentation feature dictionary stored in the temporal attention module, combined with the local detail feature map and the global semantic feature map, to obtain the information region positioning result including:

[0133] Performing online clustering of historical region segmentation features according to semantic similarity to obtain multiple feature categories, and storing the multiple feature categories in a historical region segmentation feature dictionary through a temporal attention module, wherein each feature category includes a category feature prototype and an intra-category feature variance, and the category feature prototype is weighted based on a dynamic update rate and an input historical region segmentation feature to obtain a first feature representation, and the dynamic update rate is calculated based on the intra-category feature variance, and a new feature category is created in the historical region segmentation feature dictionary when the distance between the input historical region segmentation feature and the category feature prototype is greater than a preset threshold;

[0134] Performing a non-local attention operation on the local detail feature map of the current frame to obtain a long-range dependency feature map, performing a channel attention operation on the global semantic feature map of the current frame to obtain a channel weighted feature map, performing feature matching on the long-range dependency feature map and the channel weighted feature map with the category feature prototype to obtain a first matching score and a second matching score, and combining them to obtain a feature matching score;

[0135] Calculating a first feature fusion weight and a second feature fusion weight based on the feature matching score, the intra-class feature variance and a temporal distance decay factor, wherein the temporal distance decay factor is inversely proportional to the time interval;

[0136] Performing a weighted combination of the first feature fusion weight and the long-range dependency feature map to obtain a first weighted feature map, performing a weighted combination of the second feature fusion weight and the channel weighted feature map to obtain a second weighted feature map, and combining the first weighted feature map and the second weighted feature map to obtain a feature fusion map;

[0137] Applying a spatiotemporal smoothness constraint to the feature fusion map to obtain a target feature map, wherein the spatiotemporal smoothness constraint includes a feature difference constraint item of adjacent time series frames and a feature gradient constraint item;

[0138] Based on the target feature map, region positioning is performed to obtain an information region positioning result.

[0139] The information region localization method of the dynamic memory update network is used to accurately locate specific information regions in videos, such as locating moving targets in surveillance videos or locating lesion areas in medical images. The core of this method is to build a dynamically updated historical region segmentation feature dictionary and combine the local and global features of the current frame to achieve accurate information region localization.

[0140] In an optional specific implementation, first, the regional segmentation features extracted from the historical video frames are clustered online. For example, the K-means algorithm is used to classify feature vectors with similar semantic information into one category. The number of clusters can be dynamically adjusted according to actual conditions, for example, 10 categories are initially set. Each category contains a category feature prototype and an intra-class feature variance. The category feature prototype represents the central feature of the category, which can be randomly selected initially and will be dynamically updated later. The intra-class feature variance measures the degree of discreteness of the internal features of the category, for example, calculating the average distance from each feature vector to the category feature prototype.

[0141] Next, the multiple feature categories obtained by clustering are stored in the historical region segmentation feature dictionary and managed by the temporal attention module. The temporal attention module can assign different weights to feature categories in different time periods, for example, the category closer to the current time has a higher weight. The update of each feature category depends on a dynamic update rate. This dynamic update rate is associated with the intra-class feature variance. The larger the variance, the higher the update rate. For example, if the variance of a category is large, it means that the internal features of the category are quite different and need to be updated more quickly to adapt to the new feature input. When a new historical region segmentation feature is input, its distance from the existing category feature prototype is calculated. If the distance is less than a preset threshold, such as 0.5, the feature is classified into the category and the category feature prototype is updated according to the dynamic update rate. For example, the new feature is weighted averaged with the original category feature prototype with a weight of 0.2 to obtain a new category feature prototype. If the distance is greater than the preset threshold, a new feature category is created in the dictionary and the feature is used as the initial prototype of the new category.

[0142] Then, the features of the current frame are processed. A non-local attention operation is performed on the local detail feature map of the current frame to capture long-range dependencies and obtain a long-range dependency feature map. For example, a self-attention mechanism is used to calculate the relationship between each position in the feature map and all other positions, thereby obtaining a feature representation containing global context information. A channel attention operation is performed on the global semantic feature map of the current frame to highlight important channels and obtain a channel-weighted feature map. For example, the global average pooling value of each channel is calculated, and then the weight of each channel is obtained through a fully connected layer and a sigmoid function. The long-range dependency feature map and the channel-weighted feature map are respectively matched with the feature prototypes of each category in the historical region segmentation feature dictionary, such as calculating the cosine similarity to obtain a first matching score and a second matching score. The two scores are combined, such as linearly weighted, to obtain the final feature matching score.

[0143] Based on the feature matching score, intra-class feature variance and temporal distance decay factor, the first feature fusion weight and the second feature fusion weight are calculated. The temporal distance decay factor is inversely proportional to the time interval, for example, an exponential decay function can be used. The longer the time interval, the smaller the decay factor. The first feature fusion weight is weightedly combined with the long-range dependency feature map to obtain a first weighted feature map. The second feature fusion weight is weightedly combined with the channel weighted feature map to obtain a second weighted feature map. The first weighted feature map and the second weighted feature map are combined, for example, directly added, to obtain a feature fusion map.

[0144] Apply spatiotemporal smoothing constraints to the feature fusion map to obtain the target feature map. The spatiotemporal smoothing constraints include feature difference constraints of adjacent time frames and feature gradient constraints. For example, spatiotemporal smoothing is achieved by minimizing the difference between the feature map of the current frame and the feature maps of the previous and next frames, and minimizing the gradient of the feature map. Finally, the region is located based on the target feature map, for example, using threshold segmentation or other segmentation methods to obtain the information region location result. For example, the pixels in the target feature map that are above a certain threshold are marked as information regions.

[0145] For example, assume that the size of the local detail feature map of the current frame is 256×256, and the size of the global semantic feature map is 64×64. There are 5 categories in the historical feature dictionary. The feature fusion map is calculated through the above steps, and the size is also 256×256. After the spatiotemporal smoothing constraint, the target feature map is obtained. Assuming that the pixel values ​​of the 10×10 area in the upper left corner of the target feature map are all higher than the preset threshold, the area is located as the information area.

[0146] In this embodiment, by combining historical information and spatiotemporal context information, the information area can be located more accurately, and a high positioning accuracy can be maintained even when the target appearance changes or there is occlusion; the dynamically updated historical feature dictionary can adapt to changes in the scene and improve the robustness of the method. The temporal attention module and spatiotemporal smoothing constraints also help reduce the impact of noise and interference; the online clustering and dynamic update mechanism avoids storing all historical features, reducing storage and computing costs and improving the efficiency of the method.

[0147] In an optional embodiment, multiple information sub-images are input into a multi-branch feature enhancement network, and a self-attention enhancement unit of the multi-branch feature enhancement network first extracts a feature distribution map of a character region, and adaptively enhances contrast based on the feature distribution map to obtain a feature enhanced image; the feature enhanced image is subjected to multi-model integrated recognition via a character association graph network constructed by a contextual reasoning unit to obtain an initial text recognition result; when the recognition credibility is lower than a preset threshold, the parameters of the self-attention enhancement unit are dynamically adjusted by a reinforcement learning controller for re-recognition until the credibility reaches the preset threshold, and the output of the final recognition result includes:

[0148] Input multiple information sub-images into a multi-branch feature enhancement network, divide the multiple information sub-images into multiple regular grid units, perform multi-scale morphological analysis on the regular grid units to obtain local structural features, perform wavelet transform on the local structural features to obtain feature maps of different frequency components, perform adaptive weighted fusion on the feature maps of different frequency components, and construct a character region feature distribution map; calculate the feature difference between each region and the corresponding adjacent region in the character region feature distribution map to obtain a regional enhancement coefficient, perform adaptive contrast enhancement on the multiple information sub-images at different frequency components based on the regional enhancement coefficient, and reconstruct a feature enhanced image through inverse wavelet transform;

[0149] The feature-enhanced image is input into a context reasoning unit, the context reasoning unit constructs a character association graph network based on the spatial position relationship of the feature-enhanced image, extracts structural association features and semantic association features between characters from the character association graph network, and performs Fourier transform respectively to obtain frequency domain features, calculates feature phase consistency based on the frequency domain features to obtain a cross-scale feature matching graph, and performs multi-level feature fusion on the cross-scale feature matching graph with the structural association features and the semantic association features to obtain fused features;

[0150] The fusion features are input into a pre-trained text recognition model group to perform multi-model integrated recognition. In the multi-model integrated recognition process, a character transition probability matrix is ​​constructed based on a conditional random field. The results of the multi-model integrated recognition are sequence-corrected according to the transition probability matrix to obtain multiple groups of character sequence recognition results and corresponding recognition credibility. The multiple groups of character sequence recognition results are probability-weighted using a softmax function with an adaptive temperature coefficient to obtain an initial text recognition result and a corresponding comprehensive credibility.

[0151] When the comprehensive credibility is lower than a preset credibility threshold, a reward function is constructed based on the comprehensive credibility through a reinforcement learning controller, a regularization constraint term based on prior knowledge of character structure is introduced into the reward function, and a strategy iteration algorithm with momentum term is adopted to calculate the adjustment direction of feature enhancement parameters, and the network parameters in the multi-branch feature enhancement network are simultaneously updated according to the adjustment direction to regenerate a new feature enhanced image; the iteration is repeated until the comprehensive credibility is higher than the preset credibility threshold, and the final recognition result is output.

[0152] The information sub-image text recognition method comprises the following steps:

[0153] First, multiple information sub-images to be recognized are obtained. For example, multiple character sub-images containing multiple interference factors such as blur, deformation, noise, etc. are obtained, each sub-image is 64×64 pixels in size and contains one character. These sub-images can come from scanned documents, photos, or other image sources.

[0154] Then, these information sub-images are input into a multi-branch feature enhancement network. The network divides each sub-image into multiple regular grid cells, for example, each 64×64 pixel sub-image is divided into 8×8 cells, each cell size is 8×8 pixels. Multi-scale morphological analysis is performed on each grid cell, such as using structural elements of different sizes (for example, square structural elements of 3×3, 5×5, 7×7) to perform corrosion and expansion operations to extract local structural features. These local structural features are subjected to wavelet transformation, such as using Haar wavelet for three-level decomposition to obtain feature maps of different frequency components, such as low-frequency approximate components, horizontal high-frequency detail components, vertical high-frequency detail components, and diagonal high-frequency detail components. According to the contribution of different frequency component feature maps to character information, they are adaptively weighted and fused. For example, according to the characteristics that low-frequency components contain main shape information and high-frequency components contain edge detail information, different weights are assigned to different components, and finally a feature distribution map of the character region is constructed.

[0155] Next, the regional enhancement coefficient is calculated based on the feature difference between each region and the adjacent regions in the character region feature distribution map. For example, the pixel value difference between each 8×8 pixel region and the eight adjacent regions around it is calculated and normalized to obtain the regional enhancement coefficient. Based on these regional enhancement coefficients, adaptive contrast enhancement is performed on multiple information sub-images at different frequency components. For example, the regional enhancement coefficient is multiplied by the pixel value of the corresponding region to enhance the region with obvious feature difference and suppress the region with flat feature. Then, an inverse wavelet transform is performed, such as using the Haar inverse wavelet transform, to reconstruct the feature enhanced image.

[0156] The feature-enhanced image is input to the contextual reasoning unit. The unit constructs a character association graph network based on the spatial position relationship of the feature-enhanced image. For example, each character is taken as a node, and the distance and similarity between characters are used as the weight of the edge to construct a graph network. Structural association features (e.g., relative position and distance between characters) and semantic association features (e.g., co-occurrence probability and semantic similarity between characters) between characters are extracted from the character association graph network. Fourier transform is performed on these features respectively to obtain frequency domain features. Feature phase consistency is calculated based on frequency domain features to obtain a cross-scale feature matching graph. Multi-level feature fusion is performed on the cross-scale feature matching graph with structural association features and semantic association features, for example, these features are spliced ​​together, or fused using weighted average to obtain fused features.

[0157] The fused features are input into a set of pre-trained text recognition models for multi-model integrated recognition. For example, three different convolutional neural network models are used for recognition. In the multi-model integrated recognition process, a character transition probability matrix is ​​constructed based on the conditional random field. For example, a transition probability matrix is ​​constructed based on the frequency and contextual relationship of the characters in the text. The results of the multi-model integrated recognition are sequence-corrected according to the transition probability matrix to obtain multiple sets of character sequence recognition results and corresponding recognition credibility. The softmax function with an adaptive temperature coefficient is used to probability-weight the multiple sets of character sequence recognition results to obtain the initial text recognition results and the corresponding comprehensive credibility.

[0158] If the comprehensive credibility is lower than the preset threshold, for example, lower than 0.9, a reward function is constructed based on the comprehensive credibility through a reinforcement learning controller. A regularization constraint based on prior knowledge of character structure is introduced into the reward function, for example, to penalize the situation that the recognition result does not conform to the character structure rule. A strategy iteration algorithm with momentum term is used to calculate the adjustment direction of the feature enhancement parameters. The network parameters in the multi-branch feature enhancement network are simultaneously updated according to the adjustment direction, and a new feature enhancement image is regenerated. Repeat the iteration until the comprehensive credibility is higher than the preset threshold, and the final recognition result is output.

[0159] In this embodiment, through a multi-branch feature enhancement network and a contextual reasoning unit, the local structural features, semantic association features and contextual information of characters are effectively extracted and utilized, which significantly improves the accuracy of text recognition, especially in complex scenarios such as blur, deformation, and noise. Through adaptive contrast enhancement and multi-model integrated recognition technology, the dependence on the quality of the input image is effectively reduced, the robustness of text recognition is enhanced, and it can adapt to different types of image inputs. Through the reinforcement learning controller, the feature enhancement parameters are dynamically adjusted to achieve adaptive optimization of different recognition scenarios, further improve the recognition accuracy and efficiency, and reduce the need for manual intervention.

[0160] Figure 2 FIG. 1 is a schematic diagram of the structure of an automatic ID card cutting and OCR recognition system based on AI according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0161] The first unit is used to collect multi-angle depth image data and visible light image data based on a depth camera array, fuse the multi-angle depth image data through a three-dimensional point cloud registration network, reconstruct a three-dimensional model of the ID card surface, extract a feature point sequence based on the three-dimensional model, calculate spatial posture parameters through attention posture estimation, establish a nonlinear correction model using a projection matrix optimization network, perform perspective correction on the visible light image data, and obtain standardized image data;

[0162] The second unit is used to input the standardized image data into a dual-stream memory neural network, and the dual-stream memory neural network includes a feature extraction backbone network and a dynamic memory update network; the feature extraction backbone network extracts a local detail feature map and a global semantic feature map through a multi-scale feature pyramid structure; the dynamic memory update network performs feature matching fusion based on the historical region segmentation feature dictionary stored in the temporal attention module, combines the local detail feature map and the global semantic feature map, obtains the information region positioning result, and based on the information region positioning result, divides the standardized image data to obtain multiple information sub-images, and updates the historical region segmentation feature dictionary through comparative learning;

[0163] The third unit is used to input multiple information sub-images into a multi-branch feature enhancement network, wherein the self-attention enhancement unit of the multi-branch feature enhancement network first extracts the feature distribution map of the character area, and performs adaptive contrast enhancement based on the feature distribution map to obtain a feature enhanced image; the feature enhanced image is subjected to multi-model integrated recognition via a character association graph network constructed by a context reasoning unit to obtain an initial text recognition result; when the recognition credibility is lower than a preset threshold, the parameters of the self-attention enhancement unit are dynamically adjusted through a reinforcement learning controller to re-recognize until the credibility reaches the preset threshold, and a final recognition result is output.

[0164] According to a third aspect of the embodiments of the present invention,

[0165] An electronic device is provided, comprising:

[0166] processor;

[0167] a memory for storing processor-executable instructions;

[0168] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0169] A fourth aspect of the embodiments of the present invention is:

[0170] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0171] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. The AI-based automatic ID card cutting and OCR recognition method is characterized by: include: Based on the depth camera array, multi-angle depth image data and visible light image data are collected, and the multi-angle depth image data are fused through the three-dimensional point cloud registration network to reconstruct the three-dimensional model of the ID card surface, extract the feature point sequence based on the three-dimensional model, calculate the spatial posture parameters through attention posture estimation, and use the projection matrix optimization network to establish a nonlinear correction model, perform perspective correction on the visible light image data, and obtain standardized image data; The standardized image data is input into a dual-stream memory neural network, and the dual-stream memory neural network includes a feature extraction backbone network and a dynamic memory update network; the feature extraction backbone network extracts a local detail feature map and a global semantic feature map through a multi-scale feature pyramid structure; the dynamic memory update network performs feature matching fusion based on the historical region segmentation feature dictionary stored in the temporal attention module, combines the local detail feature map and the global semantic feature map, obtains the information region positioning result, and based on the information region positioning result, the standardized image data is segmented to obtain multiple information sub-images, and the historical region segmentation feature dictionary is updated through comparative learning; Inputting a plurality of information sub-images into a multi-branch feature enhancement network, wherein a self-attention enhancement unit of the multi-branch feature enhancement network first extracts a feature distribution map of a character region, and performs adaptive contrast enhancement based on the feature distribution map to obtain a feature enhanced image; The feature-enhanced image is subjected to multi-model integrated recognition through a character association graph network constructed by a contextual reasoning unit to obtain an initial text recognition result; when the recognition credibility is lower than a preset threshold, the parameters of the self-attention enhancement unit are dynamically adjusted by a reinforcement learning controller to re-recognize until the credibility reaches the preset threshold, and the final recognition result is output.

2. The method according to claim 1, characterized in that: Based on the depth camera array, multi-angle depth image data and visible light image data are collected, and the multi-angle depth image data are fused through the three-dimensional point cloud registration network to reconstruct the three-dimensional model of the ID card surface, extract the feature point sequence based on the three-dimensional model, calculate the spatial posture parameters through attention posture estimation, and use the projection matrix optimization network to establish a nonlinear correction model, perform perspective correction on the visible light image data, and obtain standardized image data including: Based on a depth camera array, multi-angle depth image data and visible light image data are collected. The depth camera array consists of multiple depth cameras arranged in a ring at a preset elevation angle and a preset interval angle. Each of the depth cameras uses an adaptive exposure control algorithm to dynamically adjust exposure parameters according to scene light intensity, and realizes time alignment of multi-camera images based on a synchronous trigger signal between cameras; The multi-angle depth image data is input into a three-dimensional point cloud registration network. The three-dimensional point cloud registration network extracts multi-scale features through layered sampling and grouped convolution to obtain a point cloud local area descriptor. Based on a cross-attention mechanism, a similarity matrix of the point cloud local area descriptor is calculated to perform feature matching. A random consistency sampling algorithm is combined to screen matching pairs, and a camera transformation matrix is ​​determined. Registration is performed through an iterative closest point algorithm, and the multi-angle depth image data is fused to reconstruct a three-dimensional model of the ID card surface. Extracting three-dimensional feature points on the three-dimensional model of the ID card surface and constructing corresponding three-dimensional feature descriptors, extracting the spatial position relationship of the three-dimensional feature points through a spatial transformation network, weighting the geometric distribution, texture features and depth information of the three-dimensional feature points through an attention posture estimation network based on the spatial position relationship to obtain corresponding weighted feature descriptors, establishing a corresponding relationship between the three-dimensional model of the ID card surface and a standard template based on the weighted feature descriptors, and calculating spatial posture parameters using a perspective iteration algorithm; The spatial posture parameters are input into the projection matrix optimization network, the perspective transformation parameters are predicted through the residual structure, pixel-level spatial mapping is performed in combination with differentiable bilinear interpolation, a regularization constraint based on surface features is introduced to establish a nonlinear correction model, image detail information is retained through jump connections, the perspective transformation parameters are optimized using adversarial training, and the visible light image data is perspective corrected to obtain standardized image data.

3. The method according to claim 2, characterized in that Extracting three-dimensional feature points on the three-dimensional model of the ID card surface and constructing corresponding three-dimensional feature descriptors, extracting the spatial position relationship of the three-dimensional feature points through a spatial transformation network, weighting the geometric distribution, texture features and depth information of the three-dimensional feature points through an attention posture estimation network based on the spatial position relationship, and obtaining corresponding weighted feature descriptors, establishing a corresponding relationship between the three-dimensional model of the ID card surface and a standard template based on the weighted feature descriptors, and using a perspective iteration algorithm to calculate spatial posture parameters includes: Performing grid sampling on the three-dimensional model of the ID card surface to obtain a sampling point set, calculating the eigenvalue of the local neighborhood covariance matrix of each sampling point in the sampling point set to obtain the principal curvature, combining the principal curvature with the normal vector consistency constraint to construct a characteristic response function, and extracting three-dimensional feature points based on the characteristic response function; Based on the local neighborhood information of each of the three-dimensional feature points, determining the radius of the feature support domain according to the local curvature change, constructing a local reference coordinate system in the feature support domain, dividing the feature support domain into a plurality of voxels, extracting the geometric features, depth information and texture features in each voxel, and combining the geometric features, the depth information and the texture features to obtain a three-dimensional feature descriptor; Input the 3D feature descriptor into a spatial transformation network, extract the spatial position relationship of the 3D feature points by constructing a feature graph, wherein the feature graph uses the 3D feature points as vertices, uses the spatial connection relationship between the 3D feature points as edges, uses the 3D feature descriptors and position information as vertex features, and obtains spatial encoding features through graph attention convolution; Input the spatial coding features into an attention posture estimation network, calculate the geometric distribution weights of the three-dimensional feature points through a spatial attention module, calculate the texture feature weights and depth information weights of the three-dimensional feature points through a channel attention module, and fuse the geometric distribution weights, the texture feature weights and the depth information weights with the spatial coding features to obtain a weighted feature descriptor; The weighted feature descriptor and the standard template feature are subjected to Euclidean distance calculation to obtain a similarity matrix, a feature point matching scheme is determined, a one-to-one correspondence between the weighted feature descriptor and the standard template feature is established based on the feature point matching scheme, a corresponding point pair set is obtained, and a perspective iterative algorithm is used to calculate spatial posture parameters based on the corresponding point pair set.

4. The method according to claim 3, characterized in that The weighted feature descriptor and the standard template feature are subjected to Euclidean distance calculation to obtain a similarity matrix, a feature point matching scheme is determined, and a one-to-one correspondence between the weighted feature descriptor and the standard template feature is established based on the feature point matching scheme, and a corresponding point pair set is obtained, including: Performing Euclidean distance calculation on the weighted feature descriptor and the standard template feature to obtain a feature similarity matrix, and converting the feature similarity matrix into a feature matching matrix, wherein the rows of the feature matching matrix correspond to the weighted feature descriptors, and the columns of the feature matching matrix correspond to the standard template features; Subtract the minimum value of the corresponding row from each element in the feature matching matrix to obtain a first round update matrix, and subtract the minimum value of the corresponding column from each element in the first round update matrix to obtain a feature labeling matrix; By marking the zero elements in the feature marker matrix, the minimum number of rows and the minimum number of columns are selected to cover all zero elements. If the minimum number of rows and the minimum number of columns are equal to the order of the feature marker matrix, a feature point matching solution is obtained; if the minimum number of rows and the minimum number of columns are less than the order of the feature marker matrix, the minimum value is found in the uncovered elements, an adjustment amount is determined, and the adjustment amount is subtracted from all uncovered elements, and the process is repeated until a feature point matching solution is obtained; A one-to-one correspondence between the weighted feature descriptor and the standard template feature is established based on the feature point matching scheme to generate a corresponding point pair set.

5. The method according to claim 1, characterized in that The feature extraction backbone network extracts local detail feature maps and global semantic feature maps through a multi-scale feature pyramid structure, including: Based on the preset resolution attenuation coefficient and channel expansion coefficient, in the order of gradually decreasing resolution and gradually increasing number of channels, the standardized image data is subjected to a feature extraction backbone network to extract a multi-level feature map; The multi-level feature graphs are divided into a plurality of channel groups, and self-attention calculation is performed on the features in each channel group to obtain an intra-group attention feature. Based on a deformable convolutional network, a local spatial dependency relationship of the intra-group attention feature is calculated to obtain a spatial dependency weight, and the spatial dependency weight is weighted with the intra-group attention feature to obtain a first reconstructed feature; Calculate the mutual information between feature maps of adjacent levels to obtain a feature correlation coefficient, perform down-sampling and transmission on the first reconstructed feature based on the feature correlation coefficient to obtain a down-sampled feature, perform up-sampling and transmission on the first reconstructed feature to obtain an up-sampled feature, wherein the transmission weight decays exponentially with increasing transmission distance, and combine the down-sampled feature with the up-sampled feature to obtain a second reconstructed feature; Performing multi-head attention operations on the second reconstructed feature in the channel dimension and the spatial dimension respectively to obtain a channel dimension attention weight and a spatial dimension attention weight, weighting the channel dimension attention weight and the channel information of the second reconstructed feature to obtain a channel weighted feature, and weighting the spatial dimension attention weight and the spatial information of the second reconstructed feature to obtain a spatial weighted feature; The channel weighted features and the spatial weighted features are weightedly combined to obtain a feature fusion map, feature consistency constraints are imposed between adjacent features in the feature fusion map to obtain a local detail feature map, feature response strengths are calculated for the feature fusion map, the feature response strengths are screened based on a preset feature response threshold to obtain a high response area, and regional aggregation constraints are imposed on the high response area to obtain a global semantic feature map.

6. The method according to claim 1, characterized in that The dynamic memory update network performs feature matching fusion based on the historical region segmentation feature dictionary stored in the temporal attention module, combined with the local detail feature map and the global semantic feature map, and obtains the information region positioning result including: Performing online clustering of historical region segmentation features according to semantic similarity to obtain multiple feature categories, and storing the multiple feature categories in a historical region segmentation feature dictionary through a temporal attention module, wherein each feature category includes a category feature prototype and an intra-category feature variance, and the category feature prototype is weighted based on a dynamic update rate and an input historical region segmentation feature to obtain a first feature representation, and the dynamic update rate is calculated based on the intra-category feature variance, and a new feature category is created in the historical region segmentation feature dictionary when the distance between the input historical region segmentation feature and the category feature prototype is greater than a preset threshold; Performing a non-local attention operation on the local detail feature map of the current frame to obtain a long-range dependency feature map, performing a channel attention operation on the global semantic feature map of the current frame to obtain a channel weighted feature map, performing feature matching on the long-range dependency feature map and the channel weighted feature map with the category feature prototype to obtain a first matching score and a second matching score, and combining them to obtain a feature matching score; Calculating a first feature fusion weight and a second feature fusion weight based on the feature matching score, the intra-class feature variance and a temporal distance decay factor, wherein the temporal distance decay factor is inversely proportional to the time interval; Performing a weighted combination of the first feature fusion weight and the long-range dependency feature map to obtain a first weighted feature map, performing a weighted combination of the second feature fusion weight and the channel weighted feature map to obtain a second weighted feature map, and combining the first weighted feature map and the second weighted feature map to obtain a feature fusion map; Applying a spatiotemporal smoothness constraint to the feature fusion map to obtain a target feature map, wherein the spatiotemporal smoothness constraint includes a feature difference constraint item of adjacent time series frames and a feature gradient constraint item; Based on the target feature map, region positioning is performed to obtain an information region positioning result.

7. The method according to claim 1, characterized in that Inputting a plurality of information sub-images into a multi-branch feature enhancement network, wherein a self-attention enhancement unit of the multi-branch feature enhancement network first extracts a feature distribution map of a character region, and performs adaptive contrast enhancement based on the feature distribution map to obtain a feature enhanced image; The feature-enhanced image is subjected to multi-model integrated recognition through the character association graph network constructed by the context reasoning unit to obtain an initial text recognition result; when the recognition credibility is lower than a preset threshold, the parameters of the self-attention enhancement unit are dynamically adjusted by the reinforcement learning controller for re-recognition until the credibility reaches the preset threshold, and the final recognition result is output including: Input multiple information sub-images into a multi-branch feature enhancement network, divide the multiple information sub-images into multiple regular grid units, perform multi-scale morphological analysis on the regular grid units to obtain local structural features, perform wavelet transform on the local structural features to obtain feature maps of different frequency components, perform adaptive weighted fusion on the feature maps of different frequency components, and construct a character region feature distribution map; calculate the feature difference between each region and the corresponding adjacent region in the character region feature distribution map to obtain a regional enhancement coefficient, perform adaptive contrast enhancement on the multiple information sub-images at different frequency components based on the regional enhancement coefficient, and reconstruct a feature enhanced image through inverse wavelet transform; The feature-enhanced image is input into a context reasoning unit, the context reasoning unit constructs a character association graph network based on the spatial position relationship of the feature-enhanced image, extracts structural association features and semantic association features between characters from the character association graph network, and performs Fourier transform respectively to obtain frequency domain features, calculates feature phase consistency based on the frequency domain features to obtain a cross-scale feature matching graph, and performs multi-level feature fusion on the cross-scale feature matching graph with the structural association features and the semantic association features to obtain fused features; The fusion features are input into a pre-trained text recognition model group to perform multi-model integrated recognition. In the multi-model integrated recognition process, a character transition probability matrix is ​​constructed based on a conditional random field. The results of the multi-model integrated recognition are sequence-corrected according to the transition probability matrix to obtain multiple groups of character sequence recognition results and corresponding recognition credibility. The multiple groups of character sequence recognition results are probability-weighted using a softmax function with an adaptive temperature coefficient to obtain an initial text recognition result and a corresponding comprehensive credibility. When the comprehensive credibility is lower than a preset credibility threshold, a reward function is constructed based on the comprehensive credibility through a reinforcement learning controller, a regularization constraint term based on prior knowledge of character structure is introduced into the reward function, and a strategy iteration algorithm with momentum term is adopted to calculate the adjustment direction of feature enhancement parameters, and the network parameters in the multi-branch feature enhancement network are simultaneously updated according to the adjustment direction to regenerate a new feature enhanced image; the iteration is repeated until the comprehensive credibility is higher than the preset credibility threshold, and the final recognition result is output.

8. An AI-based automatic ID card cutting and OCR recognition system, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect multi-angle depth image data and visible light image data based on a depth camera array, fuse the multi-angle depth image data through a three-dimensional point cloud registration network, reconstruct a three-dimensional model of the ID card surface, extract a feature point sequence based on the three-dimensional model, calculate spatial posture parameters through attention posture estimation, establish a nonlinear correction model using a projection matrix optimization network, perform perspective correction on the visible light image data, and obtain standardized image data; A second unit is used to input the standardized image data into a two-stream memory neural network, wherein the two-stream memory neural network includes a feature extraction backbone network and a dynamic memory update network; The feature extraction backbone network extracts local detail feature maps and global semantic feature maps through a multi-scale feature pyramid structure; the dynamic memory update network performs feature matching and fusion based on the historical region segmentation feature dictionary stored in the temporal attention module, combines the local detail feature map and the global semantic feature map, obtains the information region positioning result, and based on the information region positioning result, divides the standardized image data to obtain multiple information sub-images, and updates the historical region segmentation feature dictionary through comparative learning; A third unit is used to input the multiple information sub-images into a multi-branch feature enhancement network, wherein the self-attention enhancement unit of the multi-branch feature enhancement network first extracts a character region feature distribution map, and performs adaptive contrast enhancement based on the feature distribution map to obtain a feature enhanced image; The feature-enhanced image is subjected to multi-model integrated recognition through a character association graph network constructed by a contextual reasoning unit to obtain an initial text recognition result; when the recognition credibility is lower than a preset threshold, the parameters of the self-attention enhancement unit are dynamically adjusted by a reinforcement learning controller to re-recognize until the credibility reaches the preset threshold, and the final recognition result is output.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images