Certificate image quality detection method and device, computer readable storage medium and electronic equipment

Abstract

The embodiment of the invention discloses a certificate image quality detection method and device, a computer readable storage medium and electronic equipment, and the method comprises the steps: presetting preset label data of image regions reflecting various preset quality problems in a certificate image sample for the certificate image sample, training an initial lightweight image segmentation model obtained by connecting an encoder containing a parallel vision Mangbar layer and a decoder by using a certificate image sample carrying the preset label data to obtain a lightweight image segmentation model; therefore, the lightweight image segmentation model can be utilized to perform image segmentation processing on the first certificate image of the target certificate of the user, so as to obtain an image segmentation result reflecting an image region with a preset quality problem in the first certificate image; and generating a quality detection result for the first certificate image according to the image segmentation result.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a document image quality detection method, device, computer-readable storage medium and electronic device. Background Art

[0002] In recent years, with the continuous development of image processing technology, service providers have begun to build different ID image quality detection models according to the quality detection needs of various ID images. By using the ID image quality detection model to detect the quality of the ID images submitted by users when obtaining services, they can decide whether to provide services to users based on the quality detection results, which is conducive to protecting the rights and interests of all parties. However, the current ID image quality detection schemes also face many problems. For example, the existing ID image quality detection models usually have large parameters, high usage costs, and low operating efficiency. In addition, their accuracy and versatility in practical applications are not satisfactory.

[0003] Based on this, how to improve the detection efficiency and reduce the implementation cost of the document image quality detection solution while ensuring the accuracy of the document image quality detection results has become an urgent problem that needs to be solved. Summary of the invention

[0004] The embodiments of this specification provide a document image quality detection method, device, computer-readable storage medium and electronic device, which can improve detection efficiency and reduce the implementation cost of the document image quality detection solution on the basis of ensuring the accuracy of the document image quality detection results, and have good versatility and practicality.

[0005] The embodiment of this specification provides a method for detecting the quality of a certificate image, comprising:

[0006] Acquire a first document image captured from a target document of a user;

[0007] Using a lightweight image segmentation model, image segmentation processing is performed on the image area with preset quality problems in the first document image to obtain an image segmentation result for the first document image; wherein the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and the encoder and the decoder both include a parallel visual mamba layer for feature extraction processing;

[0008] A quality detection result for the first ID image is generated according to the image segmentation result for the first ID image.

[0009] The embodiment of this specification also provides a training method for a lightweight image segmentation model, including:

[0010] Obtain a document image sample generated based on a target document sample;

[0011] Determining preset label data that the certificate image sample needs to carry; the preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem;

[0012] The initial lightweight image segmentation model is trained using the document image sample carrying the preset label data to obtain a lightweight image segmentation model; wherein the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and both the encoder and the decoder include parallel visual mamba layers for feature extraction processing.

[0013] The embodiment of this specification also provides a document image quality detection device, including:

[0014] An acquisition module, used to acquire a first document image collected from a target document of a user;

[0015] An image segmentation module is used to perform image segmentation processing on the image area with preset quality problems in the first document image using a lightweight image segmentation model to obtain an image segmentation result for the first document image; wherein the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and the encoder and the decoder both include a parallel visual mamba layer for feature extraction processing;

[0016] The detection result generating module is used to generate a quality detection result for the first document image according to the image segmentation result for the first document image.

[0017] The embodiment of this specification also provides a training device for a lightweight image segmentation model, including:

[0018] An acquisition module, used to acquire a document image sample generated based on a target document sample;

[0019] A determination module, used to determine the preset label data that the certificate image sample needs to carry; the preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem;

[0020] A training module is used to train an initial lightweight image segmentation model using document image samples carrying preset label data to obtain a lightweight image segmentation model; wherein the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and both the encoder and the decoder contain parallel visual mamba layers for feature extraction processing.

[0021] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned document image quality detection method and lightweight image segmentation model training method are implemented.

[0022] An embodiment of the present specification also provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above-mentioned document image quality detection method and lightweight image segmentation model training method.

[0023] In an embodiment of the present specification, preset label data of image areas reflecting various preset quality problems existing in the document image sample can be set in advance for the document image sample, and the document image sample carrying the preset label data can be used to train the initial lightweight image segmentation model obtained by connecting an encoder and a decoder including a parallel visual mamba layer to obtain a lightweight image segmentation model with high operating efficiency, good accuracy and good versatility. When the lightweight image segmentation model is used to perform quality detection processing on the first document image of the user's target document, the detection efficiency can be improved while ensuring the accuracy of the document image quality detection results.

[0024] In addition, since the lightweight image segmentation model contains a small number of parameters, on the one hand, it can reduce the model training cost, and on the other hand, it can also make it possible to deploy the lightweight image segmentation model on the user's terminal device, so that the terminal device can be used to implement the document image quality detection solution, reducing the interaction between the terminal device and the server device during the document image quality detection process. This is also conducive to further reducing the implementation cost of the document image quality detection solution and improving the real-time and practicality of the document image quality detection solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of an application scenario of a document image quality detection solution provided in an embodiment of this specification;

[0026] Figure 2 A flowchart of a method for detecting the quality of a document image provided in an embodiment of this specification;

[0027] Figure 3 A schematic diagram of a framework of a lightweight image segmentation model provided in an embodiment of this specification;

[0028] Figure 4 A schematic diagram of an image segmentation result for a first document image provided in an embodiment of this specification;

[0029] Figure 5A flowchart of a training method for a lightweight image segmentation model provided in an embodiment of this specification;

[0030] Figure 6 A schematic diagram of the structure of a document image quality detection device provided in an embodiment of this specification;

[0031] Figure 7 A schematic diagram of the structure of a lightweight image segmentation model training device provided in an embodiment of this specification;

[0032] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0034] The current ID image quality detection model usually has a large number of parameters and poor versatility, which not only requires longer training time, but also consumes more computing resources and time, making the implementation cost of this ID image quality detection solution high and the operation efficiency low, affecting the real-time performance of the ID image quality detection. In addition, since the current ID image quality detection model is usually not suitable for deployment at the terminal device on the user side, the terminal device and the server device need to interact in real time during the ID image quality detection process, which not only affects the operating efficiency of the ID image quality detection solution, but also easily affects the operating stability and reliability of the ID image quality detection solution.

[0035] See also Figure 1 , is a schematic diagram of an application scenario of a document image quality detection solution provided in an embodiment of this specification. Figure 1 As shown, preset label data reflecting image areas with various preset quality problems in the document image sample can be set in advance for the document image sample at the server device 103, and the document image sample carrying the preset label data can be used to train the lightweight image segmentation model obtained by connecting an encoder and a decoder including a parallel visual mamba layer, so as to obtain a lightweight image segmentation model with high operating efficiency, good accuracy and strong versatility.

[0036] The terminal device 102 can interact with the server device 103, so that the terminal device 102 can obtain the lightweight image segmentation model trained by the server device 103. When the user uses the terminal device 102 to obtain services, the user can use the terminal device 102 to collect the first document image of the user's target document 101, and use the lightweight image segmentation model deployed at the terminal device 102 to perform image segmentation processing on the image area with preset quality problems in the first document image to obtain an image segmentation result for the first document image, so as to generate a quality detection result for the first document image based on the image segmentation result for the first document image.

[0037] Of course, after obtaining the user's authorization to use and transmit the first ID image, the terminal device 102 may also send the first ID image to the server device 103, so as to utilize the lightweight image segmentation model at the server device 103 to perform segmentation processing on the image area with preset quality problems in the first ID image, and then generate a quality detection result for the first ID image based on the image segmentation result. There is no specific limitation on this.

[0038] See also Figure 2 , is a flow chart of a method for detecting the quality of a document image provided in an embodiment of this specification. The execution subject of the process may be a device for detecting the quality of a document image, or an application installed on the device for detecting the quality of a document image. Figure 2 The process shown in the figure is described in detail, and the document image quality detection method may specifically include the following steps:

[0039] Step 202: Acquire a first document image captured from a target document of a user.

[0040] In the embodiment of this specification, when there is a need to verify the target certificate of the user, the first certificate image of the target certificate of the user can be obtained. The preset type of the target certificate can be determined according to the actual needs of the device, for example, it can include but is not limited to: a certificate reflecting the identity of the user, a certificate reflecting the education level of the user, a certificate reflecting the professional qualification of the user, a certificate reflecting the assets of the user, etc., and no specific limitation is made to this.

[0041] Step 204, using a lightweight image segmentation model, performs image segmentation processing on the image area with preset quality problems in the first ID image to obtain an image segmentation result for the first ID image; wherein the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and both the encoder and the decoder include a parallel visual mamba layer for feature extraction processing.

[0042] In the embodiments of this specification, the Mamba model is a new type of selective structural state space model. By introducing the state space model and the selective mechanism, the Mamba model can not only dynamically filter and process information according to the content, establish long-distance dependencies, and more effectively remember or ignore the input part, but also effectively process long sequence data while maintaining linear time complexity. This can alleviate the modeling constraints of the convolutional neural network and solve the problem of high computational complexity and low computational efficiency of the Transformer network.

[0043] The Parallel Vision Mamba Layer (PVM Layer) is a model structure for processing deep features developed in the process of studying the Mamba model. Four parallel Visual State-Space blocks (VSS Blocks) are used in the Parallel Vision Mamba Layer to process features. The number of channels processed by each VSS block is one-fourth of the initial number of channels. The feature data extracted by each VSS block is aggregated by the concat method as the deep features extracted by the Parallel Vision Mamba Layer. Since the number of input channels in the Mamba model has an explosive effect on the number of parameters, and the VSS block parameters for processing one-fourth of the number of channels in the Parallel Vision Mamba Layer are 6.9% of the original VSS block parameters, it can achieve excellent model performance with fewer parameters and lower computational load while keeping the overall number of processing channels unchanged. Based on this, in the embodiment of this specification, a parallel vision mamba layer is used to build a lightweight image segmentation model for document image quality detection.

[0044] Specifically, the parallel visual mamba layer can be used to build the encoder and decoder in the lightweight image segmentation model. The encoder can use the parallel visual mamba layer to capture and extract comprehensive and effective deep features from the user's ID image, and the decoder can use the parallel visual mamba layer to map the deep features extracted by the encoder back to the original resolution on the basis of retaining abstract semantic information and spatial dimension information as much as possible. By performing image segmentation processing on the image area with preset quality problems in the high-quality ID image feature data extracted by the encoder, it is helpful to improve the accuracy of the image segmentation result.

[0045] In actual applications, the preset quality problems may include: at least one of the following problems: document obstruction, document incompleteness, lack of or presence of document anti-counterfeiting mark, document image overexposure and document image reflection. Of course, it may also include other problems determined according to actual needs, and no specific limitation is made to this.

[0046] In practical applications, the lightweight image segmentation model can be trained in advance using document image samples that carry preset label data reflecting the areas where various preset quality problems are located. This can enable the lightweight image segmentation model to have the ability to accurately segment image areas where various preset quality problems are located, which is conducive to improving the versatility of the lightweight image segmentation model and thus improving Figure 2 The versatility of the document image quality detection method in .

[0047] Step 206: Generate a quality detection result for the first certificate image according to the image segmentation result for the first certificate image.

[0048] In the embodiment of the present specification, if it is determined according to the image segmentation result for the first ID image that the first ID image does not contain an area with a preset quality problem, a quality inspection result reflecting that the first ID image has passed the inspection can be generated. If it is determined according to the image segmentation result for the first ID image that the first ID image contains an area with a preset quality problem, a quality inspection result reflecting that the first ID image has failed the inspection can be directly generated, or other ID image quality inspection strategies can be used to analyze the image segmentation result for the first ID image, so as to finally decide whether to allow the first ID image to pass the quality inspection, which has good flexibility.

[0049] Figure 2 In the method, since the lightweight image segmentation model is a model obtained by connecting an encoder and a decoder including a parallel visual mamba layer, and the parallel visual mamba layer has the characteristics of small parameter amount, low computational complexity and good performance, the lightweight image segmentation model can have the advantages of high operating efficiency and good accuracy. By using the lightweight image segmentation model to perform image segmentation processing on the first document image of the user's target document, an image segmentation result that reflects the image area with preset quality problems in the first document image with good accuracy can be efficiently generated, and then a quality detection result for the first document image with good accuracy can be generated based on the image segmentation result. On the basis of ensuring the accuracy of the document image quality detection result, the detection efficiency is improved.

[0050] based on Figure 2 The method in this specification also provides some specific implementation plans of the method, which are described below.

[0051] Optional, Figure 2 The method may further include: performing image segmentation processing on the first certificate image to obtain a plurality of second certificate images.

[0052] Correspondingly, step 206 may include: using a lightweight image segmentation model, based on the first ID image and the plurality of second ID images, performing image segmentation processing on image areas having preset quality problems, and obtaining an image segmentation result for the first ID image.

[0053] Wherein, the lightweight image segmentation model may further include: a multi-view feature extraction model; the multi-view feature extraction model may be connected to the encoder. The multi-view feature extraction model may be used to perform feature extraction processing and feature fusion processing on the first document image and the plurality of second document images to obtain feature data of the first document image. The encoder may be used to perform feature extraction processing on the feature data of the first document image to obtain feature data of the second document image. The decoder may be used to perform feature extraction processing and image segmentation processing on the feature data of the second document image to obtain an image segmentation result for the first document image.

[0054] In the embodiments of the present specification, multi-view feature extraction may refer to obtaining multiple perspectives of an image using multiple pictures or different image angles. By extracting and fusing features extracted from image data from multiple perspectives according to a specific strategy, and then subsequently using them for tasks such as classification or regression, more accurate image recognition, segmentation or classification results can be obtained.

[0055] Based on this, the lightweight image segmentation model can include a multi-view feature extraction model, which can use the multi-view feature extraction model to perform multi-view feature extraction and fusion on the first ID image and the local image of the first ID image (i.e., the second ID image), which is beneficial to improve the accuracy of the feature data of the first ID image extracted by the multi-view feature extraction model, and further improve the accuracy of the image segmentation results generated by the lightweight image segmentation model.

[0056] In practical applications, in order to ensure the comprehensiveness of the feature data of the first ID image extracted by the multi-view feature extraction model and simplify the ID image quality detection scheme, the sub-images obtained by segmenting the first ID image can be used as the second ID images according to the preset segmentation method, that is, the first ID image can be obtained by splicing the multiple second ID images obtained by segmenting the image. The preset segmentation method can be to divide the first ID image into a preset number of sub-images in at least one of the horizontal and vertical directions as the second ID image. Of course, the first ID image can also be divided into multiple sub-images with inconsistent areas, sizes, shapes, etc. as the second ID image, which has good flexibility.

[0057] Optionally, the multi-view feature extraction model may include: a first feature extraction module, a second feature extraction module and a feature fusion module based on a cross-attention mechanism; the first feature extraction module and the second feature extraction module may be respectively connected to the feature fusion module, and the feature fusion module may be connected to the encoder.

[0058] The first feature extraction module can be used to perform feature extraction processing on the first certificate image to obtain first global feature data corresponding to the first certificate image.

[0059] The second feature extraction module can be used to perform feature extraction processing on each of the second certificate images to obtain multiple local feature data corresponding to the first certificate image, and to perform splicing processing on the multiple local feature data to obtain second global feature data corresponding to the first certificate image.

[0060] The feature fusion module can be used to perform feature fusion processing on the first global feature data and the second global feature data based on a cross-attention mechanism to obtain the first document image feature data.

[0061] In the embodiments of this specification, in order to reduce the number of parameters of the constructed lightweight image segmentation model, the first feature extraction module of the multi-view feature extraction model included therein can be implemented by multiple convolutional layers. Of course, the first feature extraction module can also include a pooling layer or other existing network modules for extracting image feature data, which is not specifically limited. The first document image is input into the first feature extraction module to perform preliminary feature extraction processing on the first document image, thereby obtaining the first global feature data corresponding to the first document image.

[0062] Similarly, the second feature extraction module in the multi-view feature extraction model may include multiple convolutional layers and a fully connected layer. Of course, the second feature extraction module may also include a pooling layer or other existing network modules for extracting image feature data, which is not specifically limited. By inputting multiple second ID images into the second feature extraction module, preliminary feature extraction processing is performed on each second ID image, thereby obtaining multiple local feature data corresponding to the first ID image, and the fully connected layer may be used to splice the above local feature data as the second global feature data corresponding to the first ID image.

[0063] The cross attention mechanism can calculate the similarity between two different input sequences, assign weights to each element according to the similarity, and then use the weighted element vector as output, thereby achieving a good feature fusion effect. Based on this, a feature fusion module based on the cross attention mechanism can be set in the multi-view feature extraction model to perform feature fusion processing on the first global feature data and the second global feature data, which is conducive to improving the accuracy and effectiveness of the extracted first document image feature data.

[0064] Optionally, the multi-view feature extraction model may further include: a first Mambaformer module and a second Mambaformer module; the first feature extraction module may be connected to the feature fusion module via the first Mambaformer module, and the second feature extraction module may be connected to the feature fusion module via the second Mambaformer module.

[0065] The first Mambaformer module may be used to perform feature extraction processing on the first global feature data output by the first feature extraction module to obtain third global feature data.

[0066] The second Mambaformer module can be used to perform feature extraction processing on the second global feature data output by the second feature extraction module to obtain fourth global feature data.

[0067] The feature fusion module can be specifically used to perform feature fusion processing on the third global feature data and the fourth global feature data based on the cross-attention mechanism to obtain the first document image feature data.

[0068] In the embodiments of this specification, Mambaformer can be a hybrid model that combines the Mamba model with the Transformer attention mechanism. This combination not only retains the efficiency of Mamba in processing long sequences, but also takes advantage of the Transformer's advantage in capturing complex dependencies, thereby integrating the advantages of the two models to improve the overall learning ability and efficiency of the model. Based on this, the Mambaformer model can be used to build the first Mambaformer module and the second Mambaformer module.

[0069] By using the first Mambaformer module and the second Mambaformer module to perform feature extraction processing on the first global feature data and the second global feature data respectively, the representation effect of the extracted third global feature data and the fourth global feature data can be improved; thereby, the accuracy and effectiveness of the first document image feature data obtained after the feature fusion module fuses the third global feature data and the fourth global feature data can be improved.

[0070] In practical applications, the existing Mambaformer model may include an Embedding layer for receiving a tag coding vector and a time coding vector, so as to facilitate embedding processing of long sequence data. The first / second Mambaformer module in the present invention processes the feature map of the document image after feature embedding processing, so there is no need to perform feature embedding processing again, so that the first / second Mambaformer module may not include the Embedding layer in the existing Mambaformer model, which is beneficial to reducing the amount of model parameters.

[0071] For ease of understanding, Figure 3 A schematic diagram of the structure of a lightweight image segmentation model provided in the embodiments of this specification is shown in FIG. Figure 3 As shown, the lightweight image segmentation model may include: a multi-view feature extraction model 33, an encoder 31 and a decoder 32 connected in sequence. The multi-view feature extraction model 33 may include: a first feature extraction module 301, a second feature extraction module 302, a first Mambaformer module 304, a second Mambaformer module 305 and a feature fusion module 303 based on a cross attention mechanism. Figure 3 The connection relationship between the above model structures is also shown in FIG.

[0072] In practical applications, the feature fusion module in the multi-view feature extraction model can also be implemented by other model structures that can implement feature fusion processing (for example, a model structure that can run a vconcat function or perform a concatenate operation, etc.); and the multi-view feature extraction model may not include the first Mambaformer module and the second Mambaformer module to reduce the number of model parameters; there is no specific limitation on this.

[0073] Optionally, the encoder may include multiple encoding layers connected in sequence, the decoder may include multiple decoding layers connected in sequence and an output layer, and at least some of the encoding layers and the decoding layers may include the parallel visual mamba layer.

[0074] The first encoding layer in the encoder may be connected to the multi-view feature extraction model, and the last encoding layer in the encoder may be connected to the first decoding layer in the decoder.

[0075] After the first encoding layer of the encoder receives the first certificate image feature data, feature extraction processing can be performed on the first certificate image feature data by using each encoding layer, so that the last encoding layer outputs the second certificate image feature data.

[0076] After the first decoding layer of the decoder receives the second ID card image feature data, it can perform feature extraction processing based on the second ID card image feature data by using each decoding layer, so that the last decoding layer outputs third ID card image feature data, and use the output layer to perform image segmentation processing on the third ID card image feature data to obtain an image segmentation result for the image area in the first ID card image that has a preset quality problem.

[0077] In the embodiments of this specification, the encoder in the lightweight image segmentation model may include multiple sequentially connected encoding layers, the first encoding layer may receive the first document image feature data output by the multi-view feature extraction model and perform feature extraction processing, and the other encoding layers may process the document image feature data output by the previous encoding layer, thereby extracting the deep feature data of the document image. In practical applications, by setting a parallel visual mamba layer in one or more encoding layers, it is helpful to ensure the accuracy of the second document image feature data output by the last encoding layer in the encoder to the decoder. In addition, the encoding layer may also include a pooling layer or other network structure for feature processing; the structures of different encoding layers may be consistent or inconsistent, with good flexibility, and no specific limitation is made on this.

[0078] The decoder in the lightweight image segmentation model may include multiple decoding layers connected in sequence. The first decoding layer may perform feature extraction processing on the feature data of the second document image output by the encoder, and the other decoding layers may continue to process the feature data of the document image output by the previous decoding layer, thereby restoring the document image. The last layer of the decoder may be an output layer, which may perform image segmentation processing on the feature data of the third document image extracted by the last decoding layer, thereby obtaining an image segmentation result for the first document image. In practical applications, by setting a parallel visual mamba layer in one or more decoding layers, it is helpful to ensure the accuracy of the feature data of the document image output by each decoding layer. In addition, the decoding layer may also include a pooling layer or other network structure for feature processing; the structures of different decoding layers may be consistent or inconsistent, and there is no specific limitation on this.

[0079] In practical applications, the output layer in the decoder can be implemented using one or more convolutional layers, or can also be implemented using other network structures that can implement image segmentation processing, and there is no specific limitation on this. The types of image segmentation results for the first document image generated by the output layer can also be multiple, for example, it can include the location information of the image area with preset quality problems in the first document image, the category information of each pixel in the first document image (for example, the category of the preset quality problem to which the pixel belongs, or the category indicating that the pixel does not have the preset quality problem, etc.), or the mask information used to reflect the image area with preset quality problems in the first document image, etc., and there is no specific limitation on this.

[0080] Optionally, at least part of the encoding layer in the encoder may include a third Mambaformer module, and / or at least part of the decoding layer in the decoder may include a fourth Mambaformer module.

[0081] In the embodiments of this specification, it has been experimentally proved that the Mambaformer model performs better than the Mamba model and the Transformer model in the problem of long- and short-range time series prediction. Based on this, the Mambaformer model can also be used to build the encoding layer in the encoder and the decoding layer in the decoder in the lightweight image segmentation model, so as to further improve the model performance of the lightweight image segmentation model. In practical applications, the Mambaformer model can be directly used as a coding layer or a decoding layer, or the coding layer or the decoding layer can also include other existing model structures for feature extraction processing, without specific limitation.

[0082] Optionally, the lightweight image segmentation model may further include at least one jump connection module, any of which may be respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship. The encoder, the decoder and the jump connection module may constitute a symmetric UNet model or an asymmetric UNet model.

[0083] Among them, the decoding layer connected to any of the jump connection modules can be specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer, and the document image feature data output by the encoding layer connected to any of the jump connection modules, to obtain the required output document image feature data.

[0084] The decoding layer that is not connected to the skip connection module can be specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer to obtain the required output document image feature data.

[0085] In the embodiments of this specification, the UNet model architecture can be a deep learning architecture suitable for image segmentation tasks. The structural feature of the UNet model architecture is that it can include a symmetrical U-shaped structure, which can mainly include: an encoder (contraction path) and a decoder (expansion path); wherein the encoder part can gradually reduce the size of the feature map through downsampling operations, and at the same time increase the number of channels of the feature map to extract the features of the image; and the decoder part can gradually restore the size of the feature map through upsampling operations, and at the same time use a jump connection structure to fuse the feature maps of the corresponding size in the encoder to restore the spatial information of the image. This structure enables UNet to capture details while maintaining the contextual information of the image, so it performs well in image segmentation tasks.

[0086] Based on this, the lightweight image segmentation model can also include at least one jump connection module, which is beneficial to improve the accuracy of the image segmentation results generated by the lightweight image segmentation model. In practical applications, when the encoder and decoder in the lightweight image segmentation model contain the same number of layers and the network structure is consistent, the encoder, decoder and jump connection module in the lightweight image segmentation model can constitute a UNet model with good symmetry. Of course, the encoder, decoder and jump connection module in the lightweight image segmentation model can only roughly constitute a U-shaped network structure, but not a strict UNet model, by making the number of network layers contained in the encoder and decoder inconsistent, or making the network structures of the two inconsistent. It has good flexibility and is not specifically limited to this.

[0087] Optionally, the jump connection module may include: a spatial attention bridge submodule and a channel attention bridge submodule; wherein the encoding layer, the spatial attention bridge submodule, the channel attention bridge submodule and the decoding layer may be connected in sequence.

[0088] In the embodiments of this specification, the jump connection structure can establish a direct connection between the encoder and the decoder, so that shallow and deep features can communicate and fuse, thereby effectively solving the problem of traditional full convolutional networks losing details and contextual information in segmentation tasks through the jump connection structure.

[0089] Since the UltraLight VM-UNet model provides a way to use the Channel Attention Bridge (CAB) module and the Spatial Attention Bridge (SAB) module as skip connection paths, a good-performance multi-level and multi-scale information fusion method is achieved. Based on this, a spatial attention bridge submodule and a channel attention bridge submodule can be used to build a skip connection module in a lightweight image segmentation model. Of course, other existing skip connection models can also be used as the skip connection module in the lightweight image segmentation model, and there is no specific limitation on this.

[0090] In practical applications, when the feature scale output by the coding layer and the feature scale required to be received by the decoding layer are consistent, the two may have a preset corresponding relationship; or, according to actual needs, a preset corresponding relationship may be set between a designated coding layer and a designated decoding layer selected manually, so as to use a skip connection module to connect the coding layer and the decoding layer with the preset corresponding relationship, thereby realizing multi-level feature fusion.

[0091] Specifically, the ID card image feature data output by the encoding layer can be sequentially passed through the spatial attention bridge submodule and the channel attention bridge submodule in the jump connection module, so that the decoding layer can combine the ID card image feature data output by the channel attention bridge submodule and the ID card image feature data output by the previous decoding layer for feature processing to obtain the ID card image feature data finally generated by the decoding layer, thereby realizing the interaction and fusion between deep features and shallow features.

[0092] In practical applications, a spatial attention bridge submodule and a channel attention bridge submodule may be provided between each pair of encoding layers and decoding layers having a preset position correspondence relationship. Alternatively, according to actual needs, a spatial attention bridge submodule and a channel attention bridge submodule may be provided only between some encoding layers and decoding layers having a preset position correspondence relationship, and no specific limitation is made to this.

[0093] Optionally, the encoder, the decoder and the jump connection module can constitute an ultra-lightweight visual Mamba UNet model.

[0094] In the embodiments of this specification, the ultra-lightweight visual Mamba UNet model (UltraLight VM-UNet) introduces Mamba technology into the UNet architecture, thereby significantly reducing parameters and computational loads while maintaining good segmentation performance. In addition, the encoder and decoder in the ultra-lightweight visual Mamba UNet model also include parallel visual Mamba layers. Based on this, the visual Mamba UNet model can be directly used to build a lightweight image segmentation model, which is conducive to simplifying the model building process and ensuring model performance.

[0095] For ease of understanding, combined Figure 3 As shown in the structural diagram of the lightweight image segmentation model, it can be seen that the encoder 31 in the lightweight image segmentation model may include multiple sequentially connected encoding layers 306, and the decoder 32 may include multiple sequentially connected decoding layers 307 and an output layer 308. The lightweight image segmentation model may also include a skip connection module 34 to connect the encoding layer and the decoding layer having a preset corresponding relationship.

[0096] In practical applications, the number of encoding layers in the encoder 31, the number of decoding layers in the decoder 32, and the number of jump connection modules 34 can be set according to actual needs. Figure 3 This is only a feasible embodiment and not a specific limitation on the structure of the lightweight image segmentation model.

[0097] Optionally, step 206: generating a quality detection result for the first certificate image according to the image segmentation result for the first certificate image may include:

[0098] Based on the image segmentation result of the first document image, it is determined whether the overlap area ratio between the image area with preset quality problems in the first document image and the preset document key area reaches a preset threshold, and a determination result is obtained.

[0099] If the judgment result indicates that the preset quality problem exists in the first certificate image and the corresponding overlap area ratio reaches a preset threshold, a detection result indicating that the first certificate image fails the quality detection is generated.

[0100] In the embodiments of this specification, the target certificate may have an area where preset quality problems are allowed to appear, and an area where preset quality problems are not allowed to appear, so that according to actual needs, the area in the certificate image of the target certificate where preset quality problems are not allowed to appear can be used as the preset certificate key area. In addition, when it is determined based on the image segmentation result generated by the lightweight image segmentation model that the proportion of the overlap area between the image area with a certain preset quality problem in the first certificate image and the preset certificate key area reaches a preset threshold, the first certificate image detection is determined to be unsuccessful, which has good flexibility. Subsequently, information for instructing to re-capture the image of the user's target certificate for certificate image quality detection can also be generated, so that the user knows that he needs to re-capture the certificate image, which is conducive to improving the user experience.

[0101] Optional, Figure 2 The method may further include:

[0102] Preprocessing is performed on the first document image to obtain a preprocessed document image of a target size; the preprocessing includes: at least one of image correction processing and scale change processing.

[0103] Step 206 may include: using a lightweight image segmentation model to perform image segmentation processing on the image area with preset quality problems in the pre-processed document image to obtain an image segmentation result for the pre-processed document image.

[0104] The determining, based on the image segmentation result of the first document image, whether the overlap area ratio between the image region with preset quality problems in the first document image and the preset document key region reaches a preset threshold may include:

[0105] According to the image segmentation result of the pre-processed document image, first position information of an image region having a preset quality problem in the pre-processed document image is determined.

[0106] The second position information of the preset certificate key area in the preset certificate template under the target size is obtained; the certificate type to which the preset certificate template belongs is consistent with the certificate type to which the target certificate belongs.

[0107] According to the first position information and the second position information, it is determined whether the overlap area ratio between the image region with preset quality problems in the pre-processed document image and the preset document key region reaches a preset threshold.

[0108] In an embodiment of the present specification, when it is necessary to perform a document image quality detection on the document image of a target document, a preset document template that is consistent with the document type to which the target document belongs can be generated in advance. The preset document template can be a reference credible front view of the target document, and the preset document template can also reflect the document information that should be present at various positions. Therefore, according to actual needs, an area in the preset document template that is not allowed to have preset quality problems can be selected, and the position information of the area can be used as the second position information of the preset document key area.

[0109] By performing image correction and scale change processing on the first ID image that needs to be tested for ID image quality, the generated pre-processed ID image can be made consistent with the size, viewing angle, etc. of the preset ID template. At this time, the second position information of the preset ID key area in the preset ID template can be used as the position information of the preset ID key area in the pre-processed ID image. Therefore, the first position information of the area with preset quality problems detected in the pre-processed ID image and the second position information of the preset ID key area can be used according to the lightweight image segmentation model. By calculating the area intersection and union ratio, the overlap area ratio between the image area with preset quality problems in the pre-processed ID image and the preset ID key area can be obtained, which is convenient and quick.

[0110] In practical applications, the preset key areas of the document corresponding to different preset quality issues may also be different, so as to improve the flexibility of the document image quality detection solution. Figure 4 Give an example. Figure 4 A schematic diagram of an image segmentation result for a first document image provided in an embodiment of this specification, such as Figure 4 As shown, it is assumed that the image segmentation result indicates that the area 401 in the first ID image 41 has an ID occlusion problem, the area 402 has an ID anti-counterfeiting mark problem, and the area 403 has an ID punching problem. The preset ID key areas corresponding to the above three preset quality problems can be the overall area of ​​the ID image, the overall area of ​​the ID image, and the area where the text exists in the ID. Then, it can be decided whether to prompt the user to collect the image of the target ID again based on whether the overlap area ratio between the above areas 401-403 and their respective corresponding preset ID key areas reaches a preset threshold, thereby achieving precise control of the preset quality problems in the first ID image.

[0111] It can be understood that after generating the above-mentioned preprocessed ID image, image segmentation processing can also be performed on the preprocessed ID image to obtain multiple second ID images; at this time, a lightweight image segmentation model can be used to perform image segmentation processing on the preprocessed ID image and the second ID image, so as to obtain an image segmentation result for the preprocessed ID image, so as to determine whether to allow the first ID image to pass the quality inspection based on the image segmentation result, which is conducive to ensuring the accuracy of the ID image quality inspection.

[0112] Optional, Figure 2 The method described can be applied to a terminal device, and the terminal device can be equipped with the lightweight image segmentation model.

[0113] Based on this, step 202 may include: using the terminal device to perform image acquisition on the target certificate of the user to obtain a first certificate image.

[0114] Figure 2 The method may further include: using a terminal device to perform image segmentation processing on the first certificate image to obtain multiple second certificate images.

[0115] Step 204 may include: using a lightweight image segmentation model installed at the terminal device to perform image segmentation processing on the image area with preset quality problems in the first certificate image, and obtain an image segmentation result for the first certificate image.

[0116] Step 206 may include: using the terminal device to generate a quality detection result for the first document image based on the image segmentation result for the first document image.

[0117] In the embodiments of this specification, since the lightweight image segmentation model built using the parallel visual Mamba layer contains fewer parameters, it is possible to deploy the lightweight image segmentation model on the user's terminal device, so that the terminal device can be used to implement the document image quality detection solution by itself, so as to reduce the interaction between the terminal device and the server device during the document image quality detection process, which is conducive to further reducing the implementation cost of the document image quality detection solution and improving the real-time and practicality of the document image quality detection solution.

[0118] Based on the same idea, see Figure 5 , is a flowchart of a training method for a lightweight image segmentation model provided in an embodiment of this specification. The execution subject of the process can be a model training device, or an application installed on the model training device. Figure 5 The process shown is described in detail, and the training method of the lightweight image segmentation model may specifically include the following steps:

[0119] Step 502: Obtain a document image sample generated based on the target document sample.

[0120] In the embodiment of the present specification, the target certificate sample and the category of the target certificate mentioned in step 202 may generally be consistent, but there may be differences, which is not specifically limited.

[0121] Step 504, determining the preset label data that the certificate image sample needs to carry; the preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem.

[0122] In the embodiments of this specification, the preset label data may generally include: location information of the image area with preset quality problems in the document image sample, and identification information of the preset quality problems. For example, 0-4, ae, etc. may be used to represent document occlusion problems, document incomplete problems, document anti-counterfeiting mark problems, document image overexposure problems, and document image reflection problems, etc., without specific limitation.

[0123] Step 506, using the document image sample carrying the preset label data to train the initial lightweight image segmentation model to obtain a lightweight image segmentation model; wherein the lightweight image segmentation model may include: an encoder and a decoder connected in sequence, and the encoder and the decoder may both include a parallel visual mamba layer for feature extraction processing.

[0124] In the embodiments of this specification, in order to enable the lightweight image segmentation model to accurately identify various preset quality problems existing in the document image, the image segmentation task that can be performed by the lightweight image segmentation model can be set as a segmentation task for the image area with the preset quality problems. By setting preset label data for the document image sample to reflect the image area with various preset quality problems in the document image sample, and using the document image sample carrying the preset label data to train the initial lightweight image segmentation model, the trained lightweight image segmentation model can efficiently and accurately segment the image area with the preset quality problems in the document image to meet the requirements. Figure 2 The need for the method.

[0125] Optionally, step 502 may include: acquiring a first document image obtained by performing image acquisition on the target document sample, and performing image segmentation processing on the first document image to obtain a plurality of second document images.

[0126] Step 504 may include: determining preset label data for reflecting an image region in the first document image that has a preset quality problem.

[0127] Step 506 may include: using the initial lightweight image segmentation model, based on the first document image and the second document image, performing image segmentation processing on the image area with preset quality problems to obtain an image segmentation result for the first document image. Taking minimizing the difference between the image segmentation result and the preset label data as a training goal, adjusting the parameters of the initial lightweight image segmentation model to obtain a lightweight image segmentation model.

[0128] Wherein, the lightweight image segmentation model may further include: a multi-view feature extraction model; the multi-view feature extraction model is connected to the encoder. The multi-view feature extraction model may be used to perform feature extraction processing and feature fusion processing on the first document image and the plurality of second document images to obtain feature data of the first document image. The encoder may be used to perform feature extraction processing on the feature data of the first document image to obtain feature data of the second document image. The decoder may be used to perform feature extraction processing and image segmentation processing on the feature data of the second document image to obtain an image segmentation result for the first document image.

[0129] Optionally, the multi-view feature extraction model may include: a first feature extraction module, a second feature extraction module and a feature fusion module based on a cross attention mechanism; the first feature extraction module and the second feature extraction module may be connected to the feature fusion module respectively, and the feature fusion module may be connected to the encoder;

[0130] The first feature extraction module can be used to perform feature extraction processing on the first certificate image to obtain first global feature data corresponding to the first certificate image.

[0131] The second feature extraction module can be used to perform feature extraction processing on each of the second certificate images to obtain multiple local feature data corresponding to the first certificate image, and to perform splicing processing on the multiple local feature data to obtain second global feature data corresponding to the first certificate image.

[0132] The feature fusion module can be used to perform feature fusion processing on the first global feature data and the second global feature data based on a cross-attention mechanism to obtain the first document image feature data.

[0133] Optionally, the multi-view feature extraction model may further include: a first Mambaformer module and a second Mambaformer module; the first feature extraction module may be connected to the feature fusion module via the first Mambaformer module, and the second feature extraction module may be connected to the feature fusion module via the second Mambaformer module.

[0134] The first Mambaformer module can be used to perform feature extraction processing on the first global feature data output by the first feature extraction module to obtain third global feature data; the second Mambaformer module can be used to perform feature extraction processing on the second global feature data output by the second feature extraction module to obtain fourth global feature data; the feature fusion module can be specifically used to perform feature fusion processing on the third global feature data and the fourth global feature data based on the cross-attention mechanism to obtain the first document image feature data.

[0135] Optionally, the encoder may include multiple encoding layers connected in sequence, the decoder may include multiple decoding layers connected in sequence and an output layer, and at least part of the encoding layers may include the parallel visual mamba layer in the decoding layers.

[0136] The first encoding layer in the encoder may be connected to the multi-view feature extraction model, and the last encoding layer in the encoder may be connected to the first decoding layer in the decoder.

[0137] After the first encoding layer of the encoder receives the first certificate image feature data, feature extraction processing can be performed on the first certificate image feature data by using each encoding layer, so that the last encoding layer outputs the second certificate image feature data.

[0138] After the first decoding layer of the decoder receives the second ID card image feature data, it can perform feature extraction processing based on the second ID card image feature data by using each decoding layer, so that the last decoding layer outputs third ID card image feature data, and use the output layer to perform image segmentation processing on the third ID card image feature data to obtain an image segmentation result for the image area in the first ID card image that has a preset quality problem.

[0139] Optionally, at least part of the encoding layer may include a third Mambaformer module, and / or at least part of the decoding layer may include a fourth Mambaformer module.

[0140] Optionally, the lightweight image segmentation model may also include at least one jump connection module, any of which may be respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship; the encoder, the decoder and the jump connection module may constitute a symmetric UNet model or an asymmetric UNet model.

[0141] Among them, the decoding layer connected to any of the jump connection modules can be specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer, and the document image feature data output by the encoding layer connected to any of the jump connection modules, to obtain the required output document image feature data.

[0142] The decoding layer that is not connected to the skip connection module can be specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer to obtain the required output document image feature data.

[0143] Optionally, the jump connection module may include: a spatial attention bridge submodule and a channel attention bridge submodule; wherein the encoding layer, the spatial attention bridge submodule, and the channel attention bridge submodule may be sequentially connected to the decoding layer. The encoder, the decoder, and the jump connection module may constitute an ultra-lightweight visual Mamba UNet model.

[0144] Optionally, taking minimizing the difference between the image segmentation result and the preset label data as the training goal, adjusting the parameters of the initial lightweight image segmentation model to obtain the lightweight image segmentation model may include:

[0145] According to the preset loss, with minimizing the difference between the image segmentation result and the preset label data as the training goal, the parameters of the initial lightweight image segmentation model are adjusted to obtain the lightweight image segmentation model; wherein the preset loss includes: any one of Dice coefficient loss, region-specific loss and adaptive region-specific loss, and cross entropy loss.

[0146] In the embodiments of this specification, cross-entropy loss is a commonly used loss function in classification tasks. It can be a measure of the cross entropy between the predicted probability distribution and the true label probability distribution. Dice coefficient loss is widely used in image segmentation tasks, and can measure the overlap between the predicted segmentation area and the true segmentation area. Based on this, the above loss function can be used to adjust the parameters of the initial lightweight image segmentation model to obtain a lightweight image segmentation model.

[0147] In practical applications, since the Dice coefficient loss does not take into account the various segmentation difficulties that may occur in different regions of the image, it may affect the performance of the initial lightweight image segmentation model obtained through training. Therefore, the Dice coefficient loss can also be replaced by a region-specific loss or an adaptive region-specific loss to improve the performance of the lightweight image segmentation model obtained through training.

[0148] Among them, the region-specific loss can be obtained by dividing the image V into K different sub-regions V1, V2…V K , and calculate the Dice coefficient loss for each of them respectively, so that the sum of all sub-region Dice coefficient losses is used as the final region-specific loss. In this way, the Dice coefficient loss of each sub-region in the image only focuses on the prediction optimization of a specific sub-region and has nothing to do with the optimization of other sub-regions. Specifically, the region-specific loss L RS It can be expressed as formula (1):

[0149]

[0150] Among them, V k It can represent the kth sub-region in the document image sample, p i It can be expressed as the network prediction V k The probability that pixel i in belongs to each type of preset quality problem, g i Represents the corresponding preset label data, ∈ can be a constant.

[0151] Since the gradient of the region-specific loss is only affected by the prediction and label distribution within the sub-region, rather than the distribution of the entire image like the global Dice coefficient loss, and since the prediction and label distribution are different between sub-regions, it is possible to implicitly and automatically reduce the importance of sub-regions that can be well predicted, and apply higher weights to sub-regions that are more difficult to segment, thereby achieving region-specific weighting during training, which is beneficial to improving deep learning efficiency and trained model performance.

[0152] Optionally, in pixel-level segmentation learning, the network attempts to maximize the true positive TP predictions and minimize the false positive FP and false negative FN errors. In order to achieve a better trade-off between precision and recall performance, a more general Tversky loss function than the Dice coefficient loss was developed. Therefore, based on the Tversky loss function and similar considerations as the region-specific loss, an adaptive region-specific loss L is given here. ARS The expression formula (2)

[0153]

[0154] Among them, V k , p i , g i ,∈ has the same meaning as the same parameters in formula (1), which will not be elaborated here.

[0155] The parameters α and β can be fine-tuned during the training process to emphasize different types of regional errors and optimize network learning. Specifically, in a specific sub-region, if the FP error is greater than the FN error, the value of α should be increased to increase the penalty for the FP error and improve the precision. Similarly, the value of β will be increased to penalize the FN error and improve the recall rate. Based on this, the values ​​of parameters α and β can be determined according to the following formulas (3) and (4):

[0156]

[0157] Among them, V k , p i , g i ,∈ has the same meaning as the same parameters in Formula 1, which will not be described in detail. A and B can be preset constants.

[0158] Since the Adaptive Region-Specific Loss can not only pay more attention to different sub-regions that are difficult to predict, but also perform adaptive error penalties on each sub-region according to automatically adjusted function parameters during training, the performance of lightweight image segmentation models trained based on it can be improved; and since the Adaptive Region-Specific Loss can be applied to most segmentation learning frameworks without modifying the network architecture or changing the data preprocessing procedure, it is also more convenient to use.

[0159] because Figure 5 The lightweight image segmentation model in the embodiment of the method is Figure 2 The structure and working principle of the lightweight image segmentation model in the embodiment of the method may be consistent, so reference may be made to Figure 2 The working principle of the model in the embodiment of the method is explained to understand Figure 5 The working principle of the lightweight image segmentation model in is not described in detail here.

[0160] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 6 , is a schematic diagram of the structure of a document image quality detection device provided in an embodiment of this specification. Figure 6As shown, the document image quality detection device 6 can be implemented as all or part of the electronic device through software, hardware or a combination of both. According to some embodiments, the document image quality detection device 6 may include an acquisition module 61, an image segmentation module 62, and a detection result generation module 63, wherein:

[0161] The acquisition module 61 is used to acquire a first document image captured from a target document of a user.

[0162] The image segmentation module 62 is used to use a lightweight image segmentation model to perform image segmentation processing on the image area with preset quality problems in the first document image to obtain an image segmentation result for the first document image; wherein the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and both the encoder and the decoder include a parallel visual mamba layer for feature extraction processing.

[0163] The detection result generating module 63 is used to generate a quality detection result for the first document image according to the image segmentation result for the first document image.

[0164] Optional, Figure 6 The device may further include: an image segmentation module, configured to perform image segmentation processing on the first certificate image to obtain a plurality of second certificate images.

[0165] The image segmentation module 62 can be specifically used to: use a lightweight image segmentation model to perform image segmentation processing on image areas with preset quality problems based on the first ID image and the multiple second ID images, and obtain an image segmentation result for the first ID image.

[0166] Among them, the lightweight image segmentation model can also include: a multi-view feature extraction model; the multi-view feature extraction model can be connected to the encoder; the multi-view feature extraction model can be used to perform feature extraction processing and feature fusion processing on the first ID image and the multiple second ID images to obtain first ID image feature data; the encoder can be used to perform feature extraction processing on the first ID image feature data to obtain second ID image feature data; the decoder can be used to perform feature extraction processing and image segmentation processing on the second ID image feature data to obtain an image segmentation result for the first ID image.

[0167] Optionally, the multi-view feature extraction model may include: a first feature extraction module, a second feature extraction module and a feature fusion module based on a cross-attention mechanism; the first feature extraction module and the second feature extraction module may be respectively connected to the feature fusion module, and the feature fusion module may be connected to the encoder.

[0168] The first feature extraction module can be used to perform feature extraction processing on the first ID image to obtain first global feature data corresponding to the first ID image; the second feature extraction module can be used to perform feature extraction processing on each of the second ID images respectively to obtain multiple local feature data corresponding to the first ID image, and to perform splicing processing on the multiple local feature data to obtain second global feature data corresponding to the first ID image; the feature fusion module can be used to perform feature fusion processing on the first global feature data and the second global feature data based on a cross-attention mechanism to obtain the first ID image feature data.

[0169] Optionally, the multi-view feature extraction model may further include: a first Mambaformer module and a second Mambaformer module; the first feature extraction module may be connected to the feature fusion module through the first Mambaformer module, and the second feature extraction module may be connected to the feature fusion module through the second Mambaformer module; the first Mambaformer module may be used to perform feature extraction processing on the first global feature data output by the first feature extraction module to obtain third global feature data; the second Mambaformer module may be used to perform feature extraction processing on the second global feature data output by the second feature extraction module to obtain fourth global feature data; the feature fusion module may specifically be used to perform feature fusion processing on the third global feature data and the fourth global feature data based on the cross attention mechanism to obtain the first document image feature data.

[0170] Optionally, the encoder may include multiple encoding layers connected in sequence, and the decoder may include multiple decoding layers connected in sequence and an output layer, and at least some of the encoding layers and the decoding layers may include the parallel visual mamba layer; the first encoding layer in the encoder may be connected to the multi-view feature extraction model, and the last encoding layer in the encoder may be connected to the first decoding layer in the decoder; after the first encoding layer of the encoder receives the first document image feature data, it can perform feature extraction processing on the first document image feature data by using each of the encoding layers, so that the last encoding layer outputs the second document image feature data; after the first decoding layer of the decoder receives the second document image feature data, it can perform feature extraction processing based on the second document image feature data by using each of the decoding layers, so that the last decoding layer outputs third document image feature data, and use the output layer to perform image segmentation processing on the third document image feature data to obtain an image segmentation result for the image area in the first document image that has a preset quality problem.

[0171] Optionally, at least part of the encoding layer may include a third Mambaformer module, and / or at least part of the decoding layer may include a fourth Mambaformer module.

[0172] Optionally, the lightweight image segmentation model may also include at least one jump connection module, any of which may be respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship; the encoder, the decoder and the jump connection module may constitute a symmetric UNet model or an asymmetric UNet model.

[0173] Among them, the decoding layer connected to any of the jump connection modules can be specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer, and the document image feature data output by the encoding layer connected to any of the jump connection modules, to obtain the required output document image feature data.

[0174] The decoding layer that is not connected to the skip connection module can be specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer to obtain the required output document image feature data.

[0175] Optionally, the jump connection module may include: a spatial attention bridge submodule and a channel attention bridge submodule. The encoding layer, the spatial attention bridge submodule, and the channel attention bridge submodule may be sequentially connected to the decoding layer. The encoder, the decoder, and the jump connection module may constitute an ultra-lightweight visual Mamba UNet model.

[0176] Optional, Figure 6 The device in the embodiment can be applied to a terminal device equipped with a lightweight image segmentation model.

[0177] Optionally, the detection result generation module 63 may include: a judgment unit, configured to judge whether the overlap area ratio between the image region with preset quality problems and the preset key region of the certificate in the first certificate image reaches a preset threshold based on the image segmentation result of the first certificate image, and obtain a judgment result. The detection result generation unit is configured to generate a detection result indicating that the first certificate image has failed the quality inspection if the judgment result indicates that the preset quality problem exists in the first certificate image and the overlap area ratio reaches a preset threshold.

[0178] Figure 6 The device may further include: a preprocessing module, which is used to preprocess the first document image to obtain a preprocessed document image of a target size; the preprocessing includes: at least one of image correction processing and scale change processing.

[0179] The image segmentation module 62 can be used to: utilize a lightweight image segmentation model to perform image segmentation processing on an image region with preset quality problems in the pre-processed document image, and obtain an image segmentation result for the pre-processed document image.

[0180] The judgment unit may be specifically configured to determine, based on an image segmentation result of the pre-processed document image, first position information of an image region having a preset quality problem in the pre-processed document image.

[0181] The second position information of the preset certificate key area in the preset certificate template under the target size is obtained; the certificate type to which the preset certificate template belongs is consistent with the certificate type to which the target certificate belongs.

[0182] According to the first position information and the second position information, it is determined whether the overlap area ratio between the image region with preset quality problems in the pre-processed document image and the preset document key region reaches a preset threshold.

[0183] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 7 , is a schematic diagram of the structure of a training device for a lightweight image segmentation model provided in an embodiment of this specification. Figure 7 As shown, the training device 7 of the lightweight image segmentation model can be implemented as all or part of the electronic device through software, hardware or a combination of both. According to some embodiments, the training device 7 of the lightweight image segmentation model may include an acquisition module 71, a determination module 72, and a training module 73, wherein:

[0184] The acquisition module 71 is used to acquire a document image sample generated based on a target document sample.

[0185] The determination module 72 is used to determine the preset label data that the certificate image sample needs to carry; the preset label data is used to reflect the image area in the certificate image sample that has a preset quality problem.

[0186] The training module 73 is used to train the initial lightweight image segmentation model using the document image sample carrying the preset label data to obtain a lightweight image segmentation model; wherein the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and both the encoder and the decoder contain parallel visual mamba layers for feature extraction processing.

[0187] Optionally, the acquisition module 71 may be specifically configured to: acquire a first certificate image obtained by performing image acquisition on the target certificate sample, and perform image segmentation processing on the first certificate image to obtain a plurality of second certificate images.

[0188] The determination module 72 may be specifically configured to determine preset label data for reflecting an image region having a preset quality problem in the first certificate image.

[0189] The training module 73 may specifically include: an image segmentation unit, configured to use an initial lightweight image segmentation model to perform image segmentation processing on an image region with a preset quality problem based on the first document image and the second document image, to obtain an image segmentation result for the first document image. A parameter adjustment unit, configured to adjust parameters of the initial lightweight image segmentation model with minimizing the difference between the image segmentation result and the preset label data as a training target, to obtain the lightweight image segmentation model.

[0190] Among them, the lightweight image segmentation model can also include: a multi-view feature extraction model; the multi-view feature extraction model can be connected to the encoder; the multi-view feature extraction model can be used to perform feature extraction processing and feature fusion processing on the first ID image and the multiple second ID images to obtain first ID image feature data; the encoder can be used to perform feature extraction processing on the first ID image feature data to obtain second ID image feature data; the decoder can be used to perform feature extraction processing and image segmentation processing on the second ID image feature data to obtain an image segmentation result for the first ID image.

[0191] Optionally, the multi-view feature extraction model may include: a first feature extraction module, a second feature extraction module and a feature fusion module based on a cross-attention mechanism; the first feature extraction module and the second feature extraction module may be respectively connected to the feature fusion module, and the feature fusion module may be connected to the encoder.

[0192] The first feature extraction module can be used to perform feature extraction processing on the first document image to obtain first global feature data corresponding to the first document image. The second feature extraction module can be used to perform feature extraction processing on each of the second document images to obtain multiple local feature data corresponding to the first document image, and to perform splicing processing on the multiple local feature data to obtain second global feature data corresponding to the first document image. The feature fusion module can be used to perform feature fusion processing on the first global feature data and the second global feature data based on a cross-attention mechanism to obtain the first document image feature data.

[0193] Optionally, the multi-view feature extraction model may further include: a first Mambaformer module and a second Mambaformer module; the first feature extraction module may be connected to the feature fusion module via the first Mambaformer module, and the second feature extraction module may be connected to the feature fusion module via the second Mambaformer module. The first Mambaformer module may be used to perform feature extraction processing on the first global feature data output by the first feature extraction module to obtain third global feature data; the second Mambaformer module may be used to perform feature extraction processing on the second global feature data output by the second feature extraction module to obtain fourth global feature data; the feature fusion module may be specifically used to perform feature fusion processing on the third global feature data and the fourth global feature data based on a cross-attention mechanism to obtain the first document image feature data.

[0194] Optionally, the encoder may include a plurality of sequentially connected encoding layers, the decoder may include a plurality of sequentially connected decoding layers and an output layer, and at least some of the encoding layers and decoding layers may include parallel visual mamba layers. The first encoding layer in the encoder may be connected to a multi-view feature extraction model, and the last encoding layer in the encoder is connected to the first decoding layer in the decoder. After the first encoding layer of the encoder receives the first document image feature data, it may perform feature extraction processing on the first document image feature data by using each encoding layer, so that the last encoding layer outputs the second document image feature data. After the first decoding layer of the decoder receives the second document image feature data, it may perform feature extraction processing based on the second document image feature data by using each decoding layer, so that the last decoding layer outputs the third document image feature data, and the output layer may perform image segmentation processing on the third document image feature data to obtain an image segmentation result for an image area with a preset quality problem in the first document image.

[0195] Optionally, at least part of the encoding layer may include a third Mambaformer module, and / or at least part of the decoding layer may include a fourth Mambaformer module.

[0196] Optionally, the lightweight image segmentation model may also include at least one jump connection module, any of which may be respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship; the encoder, the decoder and the jump connection module may constitute a symmetric UNet model or an asymmetric UNet model.

[0197] Among them, the decoding layer connected to any of the jump connection modules can be specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer, and the document image feature data output by the encoding layer connected to any of the jump connection modules, to obtain the required output document image feature data.

[0198] The decoding layer that is not connected to the skip connection module can be specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer to obtain the required output document image feature data.

[0199] Optionally, the jump connection module may include: a spatial attention bridge submodule and a channel attention bridge submodule. The encoding layer, the spatial attention bridge submodule, the channel attention bridge submodule and the decoding layer may be connected in sequence.

[0200] Optionally, the parameter adjustment unit can be specifically used to: adjust the parameters of the initial lightweight image segmentation model according to a preset loss, with the training goal of minimizing the difference between the image segmentation result and the preset label data, so as to obtain the lightweight image segmentation model; wherein the preset loss includes: any one of Dice coefficient loss, region-specific loss and adaptive region-specific loss, and cross entropy loss.

[0201] The present specification also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above Figure 2 and Figure 5 For the method shown, the specific execution process can refer to the specific description of the relevant embodiments of the above method, which will not be repeated here.

[0202] The embodiments of this specification also provide Figure 8 The structural diagram of the electronic device shown in FIG. Figure 8 At the hardware level, the electronic device may include a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 2 and Figure 5 For the method shown, the specific execution process can refer to the specific description of the relevant embodiments of the above method, which will not be repeated here.

[0203] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0204] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for computer-readable storage media and Figure 8 As for the electronic device shown, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0205] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0206] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0207] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0208] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0209] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0210] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0211] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0212] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0213] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0214] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0215] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0216] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0217] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0218] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0219] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for detecting the quality of a document image, comprising: Acquire a first document image captured from a target document of a user; Using a lightweight image segmentation model, image segmentation processing is performed on the image area with preset quality problems in the first document image to obtain an image segmentation result for the first document image; wherein the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and the encoder and the decoder both include a parallel visual mamba layer for feature extraction processing; A quality detection result for the first ID image is generated according to the image segmentation result for the first ID image.

2. The method according to claim 1, further comprising: Performing image segmentation processing on the first certificate image to obtain a plurality of second certificate images; The step of using the lightweight image segmentation model to perform image segmentation processing on an image region in the first document image that has a preset quality problem, and obtaining an image segmentation result for the first document image, includes: Using a lightweight image segmentation model, based on the first ID image and the plurality of second ID images, image segmentation processing is performed on an image region with a preset quality problem to obtain an image segmentation result for the first ID image; Wherein, the lightweight image segmentation model further includes: a multi-view feature extraction model; the multi-view feature extraction model is connected to the encoder; The multi-view feature extraction model is used to perform feature extraction processing and feature fusion processing on the first certificate image and the plurality of second certificate images to obtain feature data of the first certificate image; The encoder is used to perform feature extraction processing on the first certificate image feature data to obtain second certificate image feature data; The decoder is used to perform feature extraction processing and image segmentation processing on the feature data of the second certificate image to obtain an image segmentation result for the first certificate image.

3. The method according to claim 2, wherein the multi-view feature extraction model comprises: A first feature extraction module, a second feature extraction module and a feature fusion module based on a cross attention mechanism; the first feature extraction module and the second feature extraction module are respectively connected to the feature fusion module, and the feature fusion module is connected to the encoder; The first feature extraction module is used to perform feature extraction processing on the first certificate image to obtain first global feature data corresponding to the first certificate image; The second feature extraction module is used to perform feature extraction processing on each of the second certificate images to obtain a plurality of local feature data corresponding to the first certificate image, and to perform splicing processing on the plurality of local feature data to obtain second global feature data corresponding to the first certificate image; The feature fusion module is used to perform feature fusion processing on the first global feature data and the second global feature data based on a cross-attention mechanism to obtain the first document image feature data.

4. The method according to claim 3, wherein the multi-view feature extraction model further comprises: A first Mambaformer module and a second Mambaformer module; the first feature extraction module is connected to the feature fusion module through the first Mambaformer module, and the second feature extraction module is connected to the feature fusion module through the second Mambaformer module; The first Mambaformer module is used to perform feature extraction processing on the first global feature data output by the first feature extraction module to obtain third global feature data; The second Mambaformer module is used to perform feature extraction processing on the second global feature data output by the second feature extraction module to obtain fourth global feature data; The feature fusion module is specifically used to perform feature fusion processing on the third global feature data and the fourth global feature data based on the cross-attention mechanism to obtain the first document image feature data.

5. The method according to claim 2, wherein the encoder comprises a plurality of sequentially connected encoding layers, the decoder comprises a plurality of sequentially connected decoding layers and an output layer, and at least part of the encoding layers and the decoding layers comprise the parallel visual mamba layer; The first encoding layer in the encoder is connected to the multi-view feature extraction model, and the last encoding layer in the encoder is connected to the first decoding layer in the decoder; After the first encoding layer of the encoder receives the first certificate image feature data, the encoder performs feature extraction processing on the first certificate image feature data by using each encoding layer, so that the last encoding layer outputs the second certificate image feature data; After the first decoding layer of the decoder receives the second ID image feature data, it uses each decoding layer to perform feature extraction processing based on the second ID image feature data, so that the last decoding layer outputs third ID image feature data, and uses the output layer to perform image segmentation processing on the third ID image feature data to obtain an image segmentation result for the image area in the first ID image that has a preset quality problem.

6. The method according to claim 5, wherein at least part of the coding layer comprises a third Mambaformer module, and / or At least part of the decoding layer includes a fourth Mambaformer module.

7. According to the method of claim 5, the lightweight image segmentation model further comprises at least one skip connection module, any of the skip connection modules is respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship; The encoder, the decoder and the skip connection module constitute a symmetric UNet model or an asymmetric UNet model; in, The decoding layer connected to any of the skip connection modules is specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer and the document image feature data output by the encoding layer connected to any of the skip connection modules to obtain the required output document image feature data; The decoding layer not connected to the skip connection module is specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer to obtain the required output document image feature data.

8. The method according to claim 7, wherein the jump connection module comprises: Spatial attention bridge submodule and channel attention bridge submodule; Among them, the encoding layer, the spatial attention bridge submodule, the channel attention bridge submodule and the decoding layer are connected in sequence.

9. According to the method of claim 8, the encoder, the decoder and the jump connection module constitute an ultra-lightweight visual Mamba UNet model.

10. The method according to any one of claims 1 to 9, wherein the method is applied to a terminal device, and the terminal device is equipped with the lightweight image segmentation model.

11. The method according to any one of claims 1 to 9, wherein generating a quality detection result for the first document image according to the image segmentation result for the first document image comprises: Based on the image segmentation result of the first document image, determine whether the overlap area ratio between the image region with preset quality problems in the first document image and the preset document key region reaches a preset threshold, and obtain a determination result; If the judgment result indicates that the preset quality problem exists in the first certificate image and the corresponding overlap area ratio reaches a preset threshold, a detection result indicating that the first certificate image fails the quality detection is generated.

12. The method according to claim 11, further comprising: Preprocessing the first document image to obtain a preprocessed document image of a target size; The preprocessing includes: at least one of image correction processing and scale change processing; The step of using the lightweight image segmentation model to perform image segmentation processing on an image region in the first document image that has a preset quality problem, and obtaining an image segmentation result for the first document image, includes: Using a lightweight image segmentation model, image segmentation processing is performed on the image area with preset quality problems in the pre-processed document image to obtain an image segmentation result for the pre-processed document image; The determining, based on the image segmentation result of the first document image, whether the overlap area ratio between the image region with preset quality problems in the first document image and the preset document key region reaches a preset threshold includes: Determining first position information of an image region having a preset quality problem in the pre-processed document image according to an image segmentation result of the pre-processed document image; Acquire second position information of a preset certificate key area in a preset certificate template under the target size; the certificate type to which the preset certificate template belongs is consistent with the certificate type to which the target certificate belongs; According to the first position information and the second position information, it is determined whether the overlap area ratio between the image region with preset quality problems in the pre-processed document image and the preset document key region reaches a preset threshold.

13. A training method for a lightweight image segmentation model, comprising: Obtain a document image sample generated based on a target document sample; Determining the preset label data that the certificate image sample needs to carry; The preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem; Using the document image sample carrying the preset label data to train an initial lightweight image segmentation model to obtain a lightweight image segmentation model; Among them, the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and both the encoder and the decoder contain parallel visual mamba layers for feature extraction processing.

14. The method according to claim 13, wherein obtaining the document image sample generated based on the target document sample comprises: Acquire a first document image obtained by performing image acquisition on the target document sample; Performing image segmentation processing on the first certificate image to obtain a plurality of second certificate images; The step of determining the preset label data that the certificate image sample needs to carry includes: Determining preset label data for reflecting an image region having a preset quality problem in the first certificate image; The step of training the initial lightweight image segmentation model using the document image sample carrying the preset label data to obtain the lightweight image segmentation model includes: Using the initial lightweight image segmentation model, based on the first ID image and the second ID image, image segmentation processing is performed on the image area with preset quality problems to obtain an image segmentation result for the first ID image; Taking minimizing the difference between the image segmentation result and the preset label data as a training goal, adjusting the parameters of the initial lightweight image segmentation model to obtain the lightweight image segmentation model; Wherein, the lightweight image segmentation model further includes: a multi-view feature extraction model; the multi-view feature extraction model is connected to the encoder; The multi-view feature extraction model is used to perform feature extraction processing and feature fusion processing on the first certificate image and the plurality of second certificate images to obtain feature data of the first certificate image; The encoder is used to perform feature extraction processing on the first certificate image feature data to obtain second certificate image feature data; The decoder is used to perform feature extraction processing and image segmentation processing on the feature data of the second certificate image to obtain an image segmentation result for the first certificate image.

15. The method according to claim 14, wherein the multi-view feature extraction model comprises: A first feature extraction module, a second feature extraction module and a feature fusion module based on a cross attention mechanism; the first feature extraction module and the second feature extraction module are respectively connected to the feature fusion module, and the feature fusion module is connected to the encoder; The first feature extraction module is used to perform feature extraction processing on the first certificate image to obtain first global feature data corresponding to the first certificate image; The second feature extraction module is used to perform feature extraction processing on each of the second certificate images to obtain a plurality of local feature data corresponding to the first certificate image, and to perform splicing processing on the plurality of local feature data to obtain second global feature data corresponding to the first certificate image; The feature fusion module is used to perform feature fusion processing on the first global feature data and the second global feature data based on a cross-attention mechanism to obtain the first document image feature data.

16. The method according to claim 15, wherein the multi-view feature extraction model further comprises: A first Mambaformer module and a second Mambaformer module; the first feature extraction module is connected to the feature fusion module through the first Mambaformer module, and the second feature extraction module is connected to the feature fusion module through the second Mambaformer module; The first Mambaformer module is used to perform feature extraction processing on the first global feature data output by the first feature extraction module to obtain third global feature data; The second Mambaformer module is used to perform feature extraction processing on the second global feature data output by the second feature extraction module to obtain fourth global feature data; The feature fusion module is specifically used to perform feature fusion processing on the third global feature data and the fourth global feature data based on the cross-attention mechanism to obtain the first document image feature data.

17. The method according to claim 14, wherein the encoder comprises a plurality of sequentially connected encoding layers, the decoder comprises a plurality of sequentially connected decoding layers and an output layer, and at least part of the encoding layers and the decoding layers comprise the parallel visual mamba layer; The first encoding layer in the encoder is connected to the multi-view feature extraction model, and the last encoding layer in the encoder is connected to the first decoding layer in the decoder; After the first encoding layer of the encoder receives the first certificate image feature data, the encoder performs feature extraction processing on the first certificate image feature data by using each encoding layer, so that the last encoding layer outputs the second certificate image feature data; After the first decoding layer of the decoder receives the second ID image feature data, it uses each decoding layer to perform feature extraction processing based on the second ID image feature data, so that the last decoding layer outputs third ID image feature data, and uses the output layer to perform image segmentation processing on the third ID image feature data to obtain an image segmentation result for the image area in the first ID image that has a preset quality problem.

18. The method according to claim 17, wherein at least part of the coding layer comprises a third Mambaformer module, and / or At least part of the decoding layer includes a fourth Mambaformer module.

19. According to the method of claim 17, the lightweight image segmentation model further comprises at least one skip connection module, any of the skip connection modules is respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship; The encoder, the decoder and the skip connection module constitute a symmetric UNet model or an asymmetric UNet model; in, The decoding layer connected to any of the skip connection modules is specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer and the document image feature data output by the encoding layer connected to any of the skip connection modules to obtain the required output document image feature data; The decoding layer not connected to the skip connection module is specifically used to perform feature extraction processing on the document image feature data output by the previous decoding layer to obtain the required output document image feature data.

20. The method according to claim 19, wherein the skip connection module comprises: Spatial attention bridge submodule and channel attention bridge submodule; Among them, the encoding layer, the spatial attention bridge submodule, the channel attention bridge submodule and the decoding layer are connected in sequence.

21. The method according to claim 14, wherein the step of adjusting parameters of the initial lightweight image segmentation model with minimizing the difference between the image segmentation result and the preset label data as a training objective to obtain the lightweight image segmentation model comprises: According to the preset loss, with minimizing the difference between the image segmentation result and the preset label data as the training goal, the parameters of the initial lightweight image segmentation model are adjusted to obtain the lightweight image segmentation model; wherein the preset loss includes: any one of Dice coefficient loss, region-specific loss and adaptive region-specific loss, and cross entropy loss.

22. A document image quality detection device, comprising: An acquisition module, used to acquire a first document image collected from a target document of a user; An image segmentation module is used to perform image segmentation processing on the image area with preset quality problems in the first document image using a lightweight image segmentation model to obtain an image segmentation result for the first document image; wherein the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and the encoder and the decoder both include a parallel visual mamba layer for feature extraction processing; The detection result generating module is used to generate a quality detection result for the first document image according to the image segmentation result for the first document image.

23. A training device for a lightweight image segmentation model, comprising: An acquisition module, used to acquire a document image sample generated based on a target document sample; A determination module, used to determine the preset label data that the certificate image sample needs to carry; The preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem; A training module, used to train an initial lightweight image segmentation model using the document image sample carrying the preset label data to obtain a lightweight image segmentation model; Among them, the lightweight image segmentation model includes: an encoder and a decoder connected in sequence, and both the encoder and the decoder contain parallel visual mamba layers for feature extraction processing.

24. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 21.

25. An electronic device comprising: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method as claimed in any one of claims 1 to 21.

Images