Certificate image quality detection method and device, computer readable storage medium and electronic equipment

Abstract

The embodiment of the invention discloses a certificate image quality detection method and device, a computer readable storage medium and electronic equipment, and the method comprises the steps: building an encoder and a decoder through a visual state space module in advance, a lightweight target detection model is obtained by sequentially connecting the encoder, the decoder and the detection head module; therefore, the lightweight target detection model can be utilized to detect the certificate image of the target certificate of the user, so as to generate a detection result for detecting whether the certificate image of the user has a preset quality problem or not.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a document image quality detection method, device, computer-readable storage medium and electronic device. Background Art

[0002] In recent years, with the continuous development of image processing technology, service providers have begun to build different ID image quality detection models for various ID image quality detection needs. By using the ID image quality detection model to detect the ID images submitted by users when obtaining services, and then deciding whether to provide services to users based on the quality detection results, it is beneficial to protect the rights and interests of all parties. However, the current ID image quality detection schemes also face many problems. For example, the existing ID image quality detection models usually have large parameters, high usage costs, and low operating efficiency. In addition, their accuracy and versatility in practical applications are not satisfactory.

[0003] Based on this, how to improve the detection efficiency and reduce the implementation cost of the document image quality detection solution while ensuring the accuracy of the document image quality detection results has become an urgent problem that needs to be solved. Summary of the invention

[0004] The embodiments of this specification provide a document image quality detection method, device, computer-readable storage medium and electronic device, which can improve detection efficiency and reduce the implementation cost of the document image quality detection solution on the basis of ensuring the accuracy of the document image quality detection results, and have good versatility and practicality.

[0005] The embodiment of this specification provides a method for detecting the quality of a certificate image, comprising:

[0006] Obtain the ID image of the user's target ID;

[0007] Using a lightweight target detection model, performing target detection processing on the document image to obtain a detection result of whether there is a preset quality problem in the document image;

[0008] The lightweight target detection model comprises: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder both contain a visual state space module for feature extraction processing.

[0009] The embodiment of this specification also provides a method for training a lightweight target detection model, including:

[0010] Obtain a document image sample generated based on a target document sample;

[0011] Determining preset label data that the certificate image sample needs to carry; the preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem;

[0012] Using the certificate image samples carrying the preset label data to train an initial lightweight object detection model, to obtain a lightweight object detection model for quality detection of certificate images;

[0013] The lightweight target detection model comprises: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder both contain a visual state space module for feature extraction processing.

[0014] The embodiment of this specification also provides a document image quality detection device, including:

[0015] An acquisition module, used to acquire a certificate image of a user's target certificate;

[0016] A detection module, used to perform target detection processing on the document image using a lightweight target detection model to obtain a detection result on whether there is a preset quality problem in the document image;

[0017] The lightweight target detection model comprises: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder both contain a visual state space module for feature extraction processing.

[0018] The embodiment of this specification also provides a training device for a lightweight target detection model, including:

[0019] An acquisition module, used to acquire a document image sample generated based on a target document sample;

[0020] A determination module, used to determine the preset label data that the certificate image sample needs to carry; the preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem;

[0021] A training module, used to train an initial lightweight object detection model using the document image samples carrying the preset label data to obtain a lightweight object detection model for quality detection of the document image;

[0022] The lightweight target detection model comprises: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder both contain a visual state space module for feature extraction processing.

[0023] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0024] An embodiment of the present specification also provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above method.

[0025] In the embodiments of the present specification, a visual state space module with good performance and low computational complexity can be used in advance to build an encoder and a decoder, and the encoder, decoder and detection head module are connected in sequence to serve as an initial lightweight target detection model. Also, preset label data of image areas reflecting various preset quality problems existing in the document image sample are set for the document image sample, and the initial lightweight target detection model is trained using the document image sample carrying the preset label data to obtain a lightweight target detection model with high operating efficiency, good accuracy and good versatility. By using the lightweight target detection model to perform quality detection on the document image of the user's target document, the detection efficiency can be improved on the basis of ensuring the accuracy of the document image quality detection result.

[0026] In addition, since the lightweight target detection model built using the visual state space module contains fewer parameters, on the one hand it can reduce the model training cost, and on the other hand it can also make it possible to deploy the lightweight target detection model on the user's terminal device, so that the terminal device can be used to implement the document image quality detection solution by itself, reducing the interaction between the terminal device and the server device during the document image quality detection process, which is also conducive to further reducing the implementation cost of the document image quality detection solution and improving the real-time and practicality of the document image quality detection solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of an application scenario of a document image quality detection solution provided in an embodiment of this specification;

[0028] Figure 2 A flowchart of a method for detecting the quality of a document image provided in an embodiment of this specification;

[0029] Figure 3 A schematic diagram of the structure of a lightweight target detection model provided in an embodiment of this specification;

[0030] Figure 4 A schematic diagram of a detection result for a preset quality problem existing in a document image provided in an embodiment of this specification;

[0031] Figure 5A flowchart of a method for training a lightweight target detection model provided in an embodiment of this specification;

[0032] Figure 6 A schematic diagram of the structure of a document image quality detection device provided in an embodiment of this specification;

[0033] Figure 7 A schematic diagram of the structure of a lightweight object detection model training device provided in an embodiment of this specification;

[0034] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0036] In recent years, with the continuous development of image processing technology, image quality evaluation function has attracted more and more attention. Since increasing the number of model parameters can improve the performance of the model to a certain extent, people usually build a document image quality detection model with a large number of parameters and a complex structure, and make each document image quality detection model specifically used to evaluate a document image quality problem, thereby ensuring the performance of the document image quality detection model.

[0037] Since the current ID image quality detection model has a large number of parameters and poor versatility, it not only requires longer training time, but also consumes more computing resources and time, making the implementation cost of this ID image quality detection solution high and the operation efficiency low, affecting the real-time performance of the ID image quality detection. In addition, since the current ID image quality detection model is usually not suitable for deployment at the terminal device on the user side, the terminal device and the server device need to interact in real time during the ID image quality detection process, which not only affects the operating efficiency of the ID image quality detection solution, but also easily affects the operating stability and reliability of the ID image quality detection solution.

[0038] See also Figure 1 , is a schematic diagram of an application scenario of a document image quality detection solution provided in an embodiment of this specification. Figure 1As shown, a Visual State-Space block with good performance and low computational complexity can be used in advance to build an encoder and a decoder, and the encoder, decoder and detection head module can be connected in sequence to form an initial lightweight target detection model. In addition, the server device 103 can be used to obtain a document image sample generated based on a target document sample; determine the preset label data required to be carried by the document image sample; the preset label data is used to reflect the image area in the document image sample where various preset quality problems exist; and the document image sample carrying the preset label data is used to train the initial lightweight target detection model to obtain a lightweight target detection model with high operating efficiency, good accuracy and good versatility.

[0039] The terminal device 102 can interact with the server device 103, so that the terminal device 102 can obtain the trained lightweight object detection model. When the user uses the terminal device 102 to obtain services, the user can use the terminal device 102 to collect the document image of the user's target document 101, and then use the lightweight object detection model deployed at the terminal device 102 to perform detection processing on the document image, thereby obtaining a detection result on whether there is a preset quality problem in the document image.

[0040] Of course, after obtaining the user's authorization to use and transmit the ID image, the terminal device 102 can also send the ID image to the server device 103 to use the lightweight target detection model at the server device 103 to generate a detection result on whether there are preset quality problems in the ID image. There is no specific limitation on this.

[0041] See also Figure 2 , is a flow chart of a method for detecting the quality of a document image provided in an embodiment of this specification. The execution subject of the process may be a device for detecting the quality of a document image, or an application installed on the device for detecting the quality of a document image. Figure 2 The process shown in the figure is described in detail, and the document image quality detection method may specifically include the following steps:

[0042] Step 202: Acquire the document image of the user's target document.

[0043] In the embodiment of this specification, when there is a need to verify the target certificate of the user, the certificate image of the target certificate of the user can be obtained. The preset type of the target certificate can be determined according to the actual needs of the device, for example, it can include but is not limited to: a certificate used to reflect the identity of the user, a certificate used to reflect the education of the user, a certificate used to reflect the professional qualifications of the user, a certificate used to reflect the assets of the user, etc., and no specific limitation is made to this.

[0044] Step 204, using a lightweight target detection model, performs target detection processing on the document image to obtain a detection result on whether there is a preset quality problem in the document image.

[0045] The lightweight target detection model may include: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder may both include a visual state space module for feature extraction processing.

[0046] In the embodiments of this specification, the existing document image quality model is usually a model built using a convolutional neural network (CNN) or a Transformer network. However, due to the limitation of the local receptive field, the CNN network is difficult to capture long-distance information, which affects the accuracy of the document image quality detection result; although the Transformer network performs well in global modeling, the computational complexity of the self-attention mechanism on the image size is high, which affects the efficiency of document image quality detection. Therefore, the structured space model with linear computational complexity and excellent performance has gradually attracted people's attention.

[0047] The Mamba model is a new type of selective structural state space model. The Mamba model can alleviate the modeling constraints of convolutional neural networks through global receptive fields and dynamic weighting, and it can achieve effective sequence modeling for long sequence data under linear computational complexity, so that it has better performance and higher operating efficiency than the Transformer network. The visual state space block (VSS Block) is a model structure developed in the process of studying the Mamba model. Since the visual state space module can enhance key information extraction to capture diverse features, and can also achieve excellent performance while significantly reducing parameters and computational costs, it can be used as a basic block in the model for capturing a wide range of contextual information. Based on this, in the embodiments of this specification, a visual state space module is used to build a lightweight target detection model for document image quality detection.

[0048] Among them, the lightweight target detection model can refer to a machine learning model for target detection with a small number of model parameters, a small amount of calculation or a small amount of memory usage. In practical applications, since the number of model parameters of the CNN or transformer network structure is several times the number of model parameters of the visual state space module (VSS Block), and the performance of the visual state space module is significantly better than that of CNN, transformer and other network structures of the same size, the model parameters of the lightweight target detection model built based on the visual state space module can be reduced by several to dozens of times compared to the target detection model built based on transformer, CNN, etc. In addition, the lightweight target detection model runs faster and the accuracy of the target detection results it generates is also better, thereby ensuring Figure 2 Efficient and accurate operation of the solution.

[0049] Specifically, the visual state space module can be used to build the encoder and decoder in the lightweight target detection model. The encoder can use the visual state space module to capture and extract comprehensive and effective deep features from the user's ID image, and the decoder can use the visual state space module to map the deep features extracted by the encoder back to the original resolution while retaining abstract semantic information and spatial dimension information as much as possible, so that the encoder can send the extracted high-quality ID image feature data to the detection head module to ensure the accuracy of the target detection results output by the detection head module.

[0050] In the embodiment of the present specification, the detection head module in the lightweight target detection model can be a module that uses a bounding box to perform target detection. In practical applications, during the model training phase, preset label data reflecting the image area with preset quality problems can be set for the document image sample, and the document image sample carrying the preset label data can be used to train the lightweight target detection model, so that the detection head module at the lightweight target detection model can use the image area with the preset quality problem as the detection target, and then obtain the detection result of whether the document image has the preset quality problem, which is convenient and quick.

[0051] In practical applications, the preset quality problems can be set according to actual needs, for example, they may include but are not limited to problems of document occlusion, document incompleteness, the presence of document anti-counterfeiting marks, document image overexposure, and document image reflection, etc. By setting preset label data of image areas reflecting various preset quality problems existing in the document image samples, and using the document image samples carrying the preset label data for model training, the trained lightweight object detection model can have the ability to detect a variety of preset quality problems existing in the document image, which is conducive to improving the versatility of the document image quality detection solution implemented based on the lightweight object detection model.

[0052] Figure 2 The method in the embodiment of the present invention uses a visual state space module with good performance, low computational complexity and few parameters to build an encoder and a decoder, and sequentially connects the encoder, decoder and detection head module to obtain a lightweight object detection model with high operating efficiency and good accuracy. Therefore, when the lightweight object detection model is used to perform quality detection on the document image of the user's target document, the detection efficiency can be improved while ensuring the accuracy of the document image quality detection result.

[0053] based on Figure 2 The method in this specification also provides some specific implementation plans of the method, which are described below.

[0054] In the embodiment of this specification, step 202: obtaining the document image of the user's target document may include:

[0055] A first document image is obtained by performing image acquisition on the target document.

[0056] Perform frequency domain transformation processing on the first certificate image to obtain a certificate spectrum diagram.

[0057] Correspondingly, step 204: using a lightweight target detection model to perform target detection processing on the document image to obtain a detection result on whether there is a preset quality problem in the document image may include:

[0058] A lightweight target detection model is used to perform target detection processing on the first document image and the document spectrum diagram to obtain a target detection result for reflecting an image area with a preset quality problem in the first document image.

[0059] Among them, the lightweight target detection model can also include: a multimodal feature extraction model; the multimodal feature extraction model can be connected to the encoder.

[0060] The multimodal feature extraction model can be used to perform feature extraction processing and feature fusion processing on the first ID image and the ID spectrum diagram to obtain feature data of the first ID image. The encoder can be used to perform feature extraction processing on the feature data of the first ID image to obtain feature data of the second ID image. The decoder can be used to perform feature extraction processing on the feature data of the second ID image to obtain feature data of the third ID image. The detection head module can be used to perform target detection processing on the feature data of the third ID image to obtain a target detection result for reflecting an image area with a preset quality problem in the first ID image.

[0061] In the embodiments of this specification, the service provider usually needs to perform quality detection on the document image in the spatial domain. Therefore, it is necessary to obtain a first document image obtained by capturing the image of the user's target document. The first document image can be of various types, for example, including but not limited to RGB images, grayscale images, etc.

[0062] At present, images in the spatial domain can be decomposed into different frequency components, from low frequency to high frequency, and each frequency component carries different characteristics of the image. For example, low-frequency components can represent smooth areas of the image, such as large backgrounds or slowly changing areas. The mid-frequency components can play a decisive role in the structure of the image, forming the main edge structure of the image to help distinguish different objects and shapes. High-frequency classification can form the edges and details of the image, providing information on texture, noise and other subtle features in the image. Since the image in the frequency domain can be used to assist in analyzing the quality of the document image in the spatial domain, the first document image can be transformed in the frequency domain to obtain a document spectrum diagram, and the first document image and the document spectrum diagram can be collaboratively detected using a lightweight target detection model, which is conducive to improving the accuracy of the target detection results generated to reflect the image area with preset quality problems in the first document image.

[0063] In practical applications, there may be many types of frequency domain transformation processing used when generating a document spectrum diagram, for example, including but not limited to: discrete cosine transform, Fourier transform, fast Fourier transform (FFT), wavelet transform, frequency domain filtering and other frequency domain transformation processing, etc., without specific limitation.

[0064] In the embodiments of this specification, in order to facilitate the encoder to perform deep feature extraction on multimodal images such as the first ID image in the spatial domain and the ID spectrogram in the frequency domain, a multimodal feature extraction model can be set in the lightweight target detection model. The multimodal feature extraction model can perform feature extraction processing and feature fusion processing on the first ID image and the ID spectrogram, thereby obtaining the first ID image feature data, and realizing the preliminary fusion of features carried in the ID images of multiple modalities. The multimodal feature extraction model can also input the first ID image feature data into the encoder in the lightweight target detection model to facilitate subsequent target detection processing.

[0065] Optionally, the multimodal feature extraction model may include: a first feature extraction module, a second feature extraction module, a frequency-aware cross-modal attention module and a weighted cross-modal fusion module.

[0066] The first feature extraction module and the second feature extraction module may both be connected to the frequency-aware cross-modal attention module, the frequency-aware cross-modal attention module may be connected to the weighted cross-modal fusion module, and the weighted cross-modal fusion module may be connected to the encoder.

[0067] Among them, the first feature extraction module can be used to perform feature extraction processing on the first document image to obtain feature data of the fourth document image. The second feature extraction module can be used to perform feature extraction processing on the document spectrum diagram to obtain feature data of the fifth document image. The frequency-aware cross-modal attention module can be used to perform feature fusion processing on the fourth document image feature data and the fifth document image feature data to obtain feature data of the sixth document image and feature data of the seventh document image. The weighted cross-modal fusion module can be used to perform weighted fusion processing on the sixth document image feature data and the seventh document image feature data to obtain feature data of the first document image.

[0068] In the embodiments of this specification, in order to reduce the number of parameters of the lightweight target detection model, the first feature extraction module and the second feature extraction module included therein can be implemented by multiple convolutional layers. Of course, the first feature extraction module and the second feature extraction module can also include a pooling layer or other existing network modules for extracting image feature data, which are not specifically limited. The first document image and the document spectrum map are respectively input into the first feature extraction module and the second feature extraction module to perform preliminary feature extraction processing on the first document image and the document spectrum map, thereby obtaining the fourth document image feature data and the fifth document image feature data.

[0069] In the embodiment of this specification, FCMNet is a model that can comprehensively utilize RGB images and depth maps to detect target saliency. The frequency-aware cross-modality attention module (Frequency-aware cross-modality attention, FACMA) in FCMNet is provided with two spatial frequency channel attention modules (Spatial frequency channel attention, SFCA), and these two spatial frequency channel attention modules can introduce data of another modality when performing feature extraction for RGB images and depth maps respectively, so as to interweave sufficient multimodal features and select representative features for output. It can be seen that the frequency-aware cross-modal attention module can perform fusion processing on multimodal features. Therefore, a frequency-aware cross-modal attention module can be set in the multimodal feature extraction model, so as to fuse the first feature extraction module and the second feature extraction module extracted from the first ID image in the spatial domain and the ID spectrum map in the frequency domain, so as to obtain the sixth ID image feature data and the seventh ID image feature data.

[0070] A weighted cross-modal fusion module (WCMF) is also proposed in FCMNet, which adaptively fuses the feature data of multiple modalities output by the frequency-aware cross-modal attention module by learning content-related weight maps. Based on this, in order to achieve a better feature fusion effect, a weighted cross-modal fusion module can also be set in the multimodal feature extraction model proposed in the embodiments of this specification, so as to fuse the sixth document image feature data and the seventh document image feature data output by the frequency-aware cross-modal attention module to obtain the first document image feature data required to be input into the encoder, which is conducive to improving the accuracy and effectiveness of the extracted first document image feature data.

[0071] Optionally, step 202: obtaining an ID image of a target ID of a user may further include: performing a scale transformation process on the first ID image to obtain at least one second ID image having a scale different from that of the first ID image.

[0072] Correspondingly, step 204: using a lightweight target detection model to perform target detection processing on the certificate image to obtain a detection result on whether there is a preset quality problem in the certificate image, may include: using a lightweight target detection model to perform target detection processing on the first certificate image, the second certificate image and the certificate spectrum diagram to obtain a target detection result for an image area reflecting the presence of a preset quality problem in the first certificate image.

[0073] Among them, the multimodal feature extraction model may also include: at least one third feature extraction module, and a scale integration unit in the ZoomNet model; the third feature extraction module and the first feature extraction module can both be connected to the scale integration unit, and the scale integration unit can be connected to the frequency-aware cross-modal attention module.

[0074] Any of the third feature extraction modules can be used to perform feature extraction processing on one of the second ID images to obtain the eighth ID image feature data. The scale integration unit can be used to perform feature fusion processing on the eighth ID image feature data and the fourth ID image feature data to obtain the ninth ID image feature data. The frequency-aware cross-modal attention module can be specifically used to perform feature fusion processing on the ninth ID image feature data and the fifth ID image feature data to obtain the sixth ID image feature data and the seventh ID image feature data.

[0075] In the embodiments of this specification, when performing target detection on images of different scales, feature information of different levels and granularities can be captured, which can help identify targets of different sizes, shapes and structures and improve the performance of target detection. Based on this, when performing image quality detection on a first ID image, a scale transformation process can also be performed on the first ID image to obtain one or more second ID images of different scales from the first ID image. In practical applications, any second ID image can be smaller or larger than the first ID image, and there is no specific limitation on this.

[0076] In the embodiments of this specification, the implementation method of the third feature extraction module can be consistent with the first / second feature extraction modules mentioned above, so as to use the third feature extraction module to perform preliminary feature extraction on the second ID image, thereby obtaining the eighth ID image feature data, which will not be elaborated.

[0077] In the embodiment of this specification, the scale integration unit in the ZoomNet model can explore key semantic clues at different scales and selectively aggregate image information at multiple scales, thereby enhancing feature representation, which is beneficial to improving the target detection accuracy of the ZoomNet model. Based on this, the scale integration unit in the ZoomNet model can be set in the multimodal feature extraction model to fuse the eighth document image feature data and the fourth document image feature data extracted from the second document image and the first document image, which is beneficial to improving the accuracy of the generated ninth document image feature data, thereby improving the accuracy of the document image quality detection results.

[0078] Optionally, the multimodal feature extraction model may further include: a Mambaformer module; the weighted cross-modal fusion module may be connected to the encoder through the Mambaformer module. The weighted cross-modal fusion module may be specifically used to perform weighted fusion processing on the sixth document image feature data and the seventh document image feature data to obtain the tenth document image feature data. The Mambaformer module is used to perform feature extraction processing on the tenth document image feature data to obtain the first document image feature data.

[0079] In the embodiments of this specification, Mambaformer can be a hybrid model that combines Mamba, a selective state space model, with the Transformer attention mechanism. This combination not only retains the efficiency of Mamba in processing long sequences, but also takes advantage of the Transformer's advantage in capturing complex dependencies, thereby integrating the advantages of the two models to improve the overall learning ability and efficiency of the model. Based on this, a Mambaformer module can be set in the multimodal feature extraction model. By using the Mambaformer module, the tenth document image feature data output by the weighted cross-modal fusion module is further processed to improve the accuracy and effectiveness of the first document image feature data input to the encoder, thereby ensuring the credibility of the document image quality detection results.

[0080] For ease of understanding, Figure 3 A schematic diagram of the structure of a lightweight target detection model provided in the embodiments of this specification is shown in FIG. Figure 3 As shown, the lightweight object detection model may include: a multimodal feature extraction model 34, an encoder 31, a decoder 32 and a detection head module 33 connected in sequence. Among them, the multimodal feature extraction model 34 may include: a first feature extraction module 301, a second feature extraction module 302, a third feature extraction module 305, a scale integration unit 306, a frequency-aware cross-modal attention module 303, a weighted cross-modal fusion module 304 and a Mambaformer module 307. Figure 3 The connection relationship between the above model structures is also shown in FIG.

[0081] In practical applications, structures such as the scale integration unit 306, the frequency-aware cross-modal attention module 303 and the weighted cross-modal fusion module 304 can also be implemented through other model structures that can realize multi-scale and multi-modal feature fusion processing, and no specific limitation is made to this.

[0082] Optionally, the encoder in the lightweight target detection model may include multiple encoding layers connected in sequence, the decoder may include multiple decoding layers connected in sequence, at least some of the encoding layers and the decoding layers may include the visual state space module; and the detection head module may include a target number of target detection layers.

[0083] The first encoding layer in the encoder can be connected to the multimodal feature extraction model, the last encoding layer in the encoder can be connected to the first decoding layer in the decoder, and the target number of decoding layers in the decoder can be connected one-to-one with the target number of target detection layers.

[0084] After the first encoding layer of the encoder receives the first certificate image feature data, feature extraction processing can be performed on the first certificate image feature data by using each encoding layer, so that the last encoding layer outputs the second certificate image feature data.

[0085] After the first decoding layer of the decoder receives the second certificate image feature data, each decoding layer may perform feature extraction processing based on the second certificate image feature data to obtain the third certificate image feature data output by each decoding layer.

[0086] The detection head module can perform target detection processing on the received third document image feature data by using the target number of target detection layers to obtain a target detection result reflecting an image area with a preset quality problem in the first document image.

[0087] In the embodiments of this specification, the encoder in the lightweight object detection model may include multiple sequentially connected encoding layers, the first encoding layer may receive the first document image feature data output by the multimodal feature extraction model and perform feature extraction processing, and the other encoding layers may process the document image feature data output by the previous encoding layer, thereby extracting the deep feature data of the document image. In practical applications, by setting a visual state space module in one or more encoding layers, it is helpful to ensure the accuracy of the second document image feature data output by the last encoding layer in the encoder to the decoder. In addition, the encoding layer may also include a pooling layer or other network structure for feature processing; the structures of different encoding layers may be consistent or inconsistent, and there is no specific limitation on this.

[0088] The decoder in the lightweight object detection model may include multiple decoding layers connected in sequence. The first decoding layer may perform feature extraction processing on the feature data of the second document image output by the encoder, and the other decoding layers may continue to process the feature data of the document image output by the previous decoding layer, thereby restoring the document image. In practical applications, by setting a visual state space module in one or more decoding layers, it is helpful to ensure the accuracy of the feature data of the third document image output by each decoding layer. In addition, the decoding layer may also include a pooling layer or other network structure for feature processing; the structures of different decoding layers may be consistent or inconsistent, and there is no specific limitation on this.

[0089] In the embodiments of this specification, the target detection layer in the detection head module can be implemented by using a Detect layer such as in the YOLOv series model, or can also be implemented by using other network layers that can output the location information of the detected target object, without specific limitation. In practical applications, the number of targets in the target detection layer in the detection head module can usually be less than or equal to the number of decoding layers in the decoder in the lightweight target detection model, and each target detection layer can be connected to a decoding layer, and the decoding layers connected to different target detection layers are usually different, so that the target detection layer can be used to perform target detection on the third document image feature data output by the decoding layer connected to it.

[0090] It is understandable that when the detection head module includes multiple target detection layers, the sizes of the third ID image feature data processed by different target detection layers are usually not consistent, so that the detection head module can perform target detection on feature information of different levels and granularities, which is beneficial to improving the accuracy of the target detection results it generates for the image area in the first ID image that has preset quality problems.

[0091] Optionally, the lightweight object detection model may further include: at least one skip connection module; any of the skip connection modules may be respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship.

[0092] The encoder, the decoder and the jump connection module can constitute a symmetric UNet model or an asymmetric UNet model.

[0093] Among them, the decoding layer connected to any of the jump connection modules can be used to generate the third certificate image feature data based on the certificate image feature data output by the previous decoding layer and the certificate image feature data output by the encoding layer connected to any of the jump connection modules.

[0094] The decoding layer that is not connected to the skip connection module can be used to generate the third document image feature data according to the document image feature data output by the previous decoding layer.

[0095] In the embodiments of this specification, the skip connections in the UNet model can establish a direct connection between the encoder and the decoder, so that shallow and deep features can communicate and fuse, thereby effectively solving the problem of traditional full convolutional networks losing details and contextual information in segmentation tasks through the skip connection structure. Based on this, the lightweight object detection model can also include a skip connection module. At this time, the encoder, decoder and skip connection module in the lightweight object detection model can constitute a symmetrical UNet model or an asymmetrical UNet model.

[0096] In practical applications, when the feature scale output by the coding layer and the feature scale required to be received by the decoding layer are consistent, the two can have a preset corresponding relationship; or, according to actual needs, a preset corresponding relationship can be set between a designated coding layer and a designated decoding layer selected manually. This allows the use of a skip connection module to connect the coding layer and the decoding layer with a preset corresponding relationship, which is beneficial to improving model performance.

[0097] Optionally, the encoder in the lightweight object detection model can be the encoder in the Visual Mamba UNet model.

[0098] Furthermore, the decoder in the lightweight object detection model may be a decoder in the visual Mamba UNet model.

[0099] Among them, the Visual Mamba UNet model is a lightweight U-shaped segmentation model based on Mamba, which includes an encoder and a decoder connected in sequence, and a jump connection module can also be set between the encoder and the decoder. The Visual Mamba UNet model can achieve excellent image segmentation performance while significantly reducing parameters and computing costs. Based on this, the encoder and decoder in the Visual Mamba UNet model can be directly used to build a lightweight object detection model, which is conducive to simplifying the model building process and ensuring model performance.

[0100] For ease of understanding, combined Figure 3From the structural diagram of the lightweight target detection model shown in , it can be seen that the encoder 31 in the lightweight target detection model can include multiple encoding layers connected in sequence, and the decoder 32 can include multiple decoding layers connected in sequence. The lightweight target detection model can also include a jump connection module 35 to connect the encoding layer and the decoding layer with a preset corresponding relationship. And, the detection head module 33 can also include a target detection layer, and the target detection layer is used to perform target detection on the document image feature data output by the decoding layer in the decoder 32 to obtain the document image quality detection result.

[0101] In practical applications, the number of encoding layers in the encoder 31, the number of decoding layers in the decoder 32, the number of jump connection modules 35, and the number of target detection layers in the detection head module 33 can all be set according to actual needs. Figure 3 This is only a feasible embodiment and not a specific limitation on the structure of the lightweight object detection model.

[0102] Optionally, step 204: after performing target detection processing on the document image using a lightweight target detection model and obtaining a detection result on whether there is a preset quality problem in the document image, the following may also be included:

[0103] Based on the detection result, it is determined whether the overlap area ratio between the image region with the preset quality problem in the certificate image and the preset certificate key region reaches a preset threshold, and a determination result is obtained.

[0104] If it is determined according to the judgment result that the preset quality problem exists in the document image and the corresponding overlap area ratio reaches the preset threshold, information is generated to instruct the image of the user's target document to be captured again for document image quality detection.

[0105] In the embodiments of this specification, the target certificate may have an area where preset quality problems are allowed to appear, and an area where preset quality problems are not allowed to appear, so that according to actual needs, the area in the certificate image where preset quality problems are not allowed to appear can be used as the preset certificate key area. In addition, when the detection result generated for the certificate image according to the lightweight target detection model is determined to have a certain preset quality problem in the certificate image and the overlap area ratio between the preset certificate key area reaches a preset threshold, the certificate image detection is determined to be failed, which has good flexibility. Subsequently, information for indicating that the image of the user's target certificate should be collected again for certificate image quality detection can be generated, so that the user knows that he needs to re-collect the certificate image, which is conducive to improving the user experience.

[0106] Optional, Figure 2The method may further include: performing preprocessing on the document image to obtain a preprocessed document image of a target size; the preprocessing includes: at least one of image correction processing and scale change processing.

[0107] Correspondingly, the use of a lightweight target detection model to perform target detection processing on the document image to obtain a detection result on whether there is a preset quality problem in the document image may include: using a lightweight target detection model to perform target detection processing on the pre-processed document image to obtain first position information of the image area reflecting the presence of the preset quality problem in the pre-processed document image.

[0108] Correspondingly, judging, based on the detection result, whether the overlap area ratio between the image region with the preset quality problem in the certificate image and the preset certificate key region reaches a preset threshold may include:

[0109] The second position information of the preset certificate key area in the preset certificate template under the target size is obtained; the certificate type to which the preset certificate template belongs is consistent with the certificate type to which the target certificate belongs.

[0110] According to the first position information and the second position information, it is determined whether the overlap area ratio between the image region with preset quality problems in the pre-processed document image and the preset document key region reaches a preset threshold.

[0111] In an embodiment of the present specification, when it is necessary to perform a document image quality detection on the document image of a target document, a preset document template that is consistent with the document type to which the target document belongs can be generated in advance. The preset document template can be a reference credible front view of the target document, and the preset document template can also reflect the document information that should be present at various positions. Therefore, according to actual needs, an area in the preset document template that is not allowed to have preset quality problems can be selected, and the position information of the area can be used as the second position information of the preset document key area.

[0112] By performing image correction and scale change processing on the ID image that needs to be tested for ID image quality, the generated pre-processed ID image can be made consistent with the size, viewing angle, etc. of the preset ID template. At this time, the second position information of the preset ID key area in the preset ID template can be used as the position information of the preset ID key area in the pre-processed ID image. Therefore, the first position information of the area with preset quality problems detected by the lightweight target detection model for the pre-processed ID image and the second position information of the preset ID key area can be used to obtain the proportion of the overlapping area between the image area with preset quality problems and the preset ID key area in the pre-processed ID image by calculating the area intersection and union ratio, which is convenient and quick.

[0113] In practical applications, the preset key areas of the document corresponding to different preset quality issues may also be different, so as to improve the flexibility of the document image quality detection solution. Figure 4 Give an example. Figure 4 A schematic diagram of a detection result for a preset quality problem existing in a document image provided in an embodiment of this specification, such as Figure 4 As shown, it is assumed that area 401 in the document image 41 has a document occlusion problem, area 402 has a document anti-counterfeiting mark problem, and area 403 has a document punching problem. The preset document key areas corresponding to the above three preset quality problems can be the overall area of ​​the document image, the overall area of ​​the document image, and the area where the text exists in the document. Then, by determining whether the overlap area ratio between the above areas 401-403 and their respective corresponding preset document key areas reaches a preset threshold, it is decided whether to prompt the user to collect the image of the target document again, so as to achieve precise control of the preset quality problems in the document image.

[0114] It can be understood that the preprocessed document image can be specifically obtained by preprocessing the first document image; and, the preprocessed document image can also be subjected to frequency domain transformation processing to obtain a document spectrum diagram; and, the preprocessed document image can be subjected to scale transformation processing to obtain at least one second document image having a scale different from that of the preprocessed document image, so that a lightweight target detection model can be used to perform target detection processing on the preprocessed document image, the second document image and the document spectrum diagram to obtain a target detection result for reflecting the image area with preset quality problems in the preprocessed document image, so as to judge whether the proportion of the overlapping area between the image area with preset quality problems in the preprocessed document image and the preset document key area reaches a preset threshold value according to the target detection result. If so, it can be determined that the first document image has failed the quality inspection, which is conducive to ensuring the accuracy of the document image quality inspection.

[0115] In the embodiments of this specification, Figure 2 The document image quality detection method in the present invention can be applied to a terminal device, and the terminal device can be equipped with a lightweight target detection model.

[0116] Based on this, step 202 may include: using a terminal device to obtain a first ID image obtained by performing image acquisition on the target ID; performing frequency domain transformation processing on the first ID image to obtain an ID spectrum diagram; performing scale transformation processing on the first ID image to obtain at least one second ID image with a scale different from that of the first ID image.

[0117] Step 204 may include: using a lightweight target detection model installed at the terminal device to perform target detection processing on the first ID image, the second ID image and the ID spectrum diagram, and obtain a detection result for reflecting the image area with preset quality problems in the first ID image.

[0118] besides, Figure 2 The method may also include: using the terminal device to determine, based on the detection result, whether the overlap area ratio between the image area with the preset quality problem in the certificate image and the preset certificate key area reaches a preset threshold, and obtain a judgment result.

[0119] If it is determined based on the judgment result that there is a preset quality problem in the certificate image where the corresponding overlapping area ratio reaches a preset threshold, the terminal device is used to generate and display information for instructing the user's target certificate image to be captured again for certificate image quality detection.

[0120] In the embodiments of this specification, since the lightweight target detection model built using the visual state space module contains fewer parameters, it is possible to deploy the lightweight target detection model on the user's terminal device, so that the terminal device can be used to implement the document image quality detection solution by itself, so as to reduce the interaction between the terminal device and the server device during the document image quality process, which is conducive to further reducing the implementation cost of the document image quality detection solution and improving the real-time and practicality of the document image quality detection solution.

[0121] Based on the same idea, see Figure 5 , is a flow chart of a training method for a lightweight target detection model provided in an embodiment of this specification. The execution subject of the process can be a model training device, or an application installed on the model training device. Figure 5 The process shown is described in detail, and the training method of the lightweight target detection model may specifically include the following steps:

[0122] Step 502: Obtain a document image sample generated based on the target document sample.

[0123] In the embodiment of the present specification, the target certificate sample and the category of the target certificate mentioned in step 202 may generally be consistent, but there may be differences, which is not specifically limited.

[0124] Step 504, determining the preset label data that the certificate image sample needs to carry; the preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem.

[0125] In the embodiments of this specification, the preset label data may generally include: location information of the image area with preset quality problems in the document image sample, and identification information of the preset quality problems. For example, 0-4, ae, etc. may be used to represent document occlusion problems, document incomplete problems, document anti-counterfeiting mark problems, document image overexposure problems, and document image reflection problems, etc., without specific limitation.

[0126] Step 506: train the initial lightweight object detection model using the document image sample carrying the preset label data to obtain a lightweight object detection model for quality detection of the document image. The lightweight object detection model may include: an encoder, a decoder, and a detection head module connected in sequence; the encoder and the decoder may both include a visual state space module for feature extraction processing.

[0127] In the embodiments of this specification, the lightweight object detection model built using the visual state space module contains fewer parameters and has low computational complexity, thereby reducing the model training cost. In addition, the lightweight object detection model trained using the document image samples with preset label data can accurately identify areas with various preset quality problems in the image, thereby meeting the requirements of the image quality control system. Figure 2 The demand for document image quality detection in the enterprise is conducive to improving the accuracy, effectiveness and versatility of document image quality detection.

[0128] Optionally, obtaining a document image sample generated based on the target document sample may include:

[0129] A first document image is obtained by performing image acquisition on the target document sample.

[0130] Perform frequency domain transformation processing on the first certificate image to obtain a certificate spectrum diagram.

[0131] Correspondingly, the step of determining the preset label data that the certificate image sample needs to carry may include:

[0132] Determine preset label data for reflecting an image region having a preset quality problem in the first document image.

[0133] The step of training the initial lightweight object detection model using the document image sample carrying the preset label data to obtain the lightweight object detection model for quality detection of the document image may include:

[0134] An initial lightweight target detection model is used to perform target detection processing on the first document image and the document spectrum diagram to obtain a prediction result for an image area in the first document image that has the preset quality problem.

[0135] Taking minimizing the difference between the prediction result and the preset label data as the training goal, the parameters of the initial lightweight object detection model are adjusted to obtain the lightweight object detection model.

[0136] Among them, the lightweight target detection model can also include: a multimodal feature extraction model; the multimodal feature extraction model is connected to the encoder.

[0137] The multimodal feature extraction model can be used to perform feature extraction processing and feature fusion processing on the first ID image and the ID spectrum diagram to obtain feature data of the first ID image. The encoder can be used to perform feature extraction processing on the feature data of the first ID image to obtain feature data of the second ID image. The decoder can be used to perform feature extraction processing on the feature data of the second ID image to obtain feature data of the third ID image. The detection head module can be used to perform target detection processing on the feature data of the third ID image to obtain a prediction result for an image area in the first ID image that has the preset quality problem.

[0138] Optionally, the multimodal feature extraction model may include: a first feature extraction module, a second feature extraction module, a frequency-aware cross-modal attention module and a weighted cross-modal fusion module.

[0139] Both the first feature extraction module and the second feature extraction module can be connected to the frequency-aware cross-modal attention module, the frequency-aware cross-modal attention module can be connected to the weighted cross-modal fusion module, and the weighted cross-modal fusion module can be connected to the encoder.

[0140] Among them, the first feature extraction module can be used to perform feature extraction processing on the first document image to obtain feature data of the fourth document image. The second feature extraction module can be used to perform feature extraction processing on the document spectrum diagram to obtain feature data of the fifth document image. The frequency-aware cross-modal attention module can be used to perform feature fusion processing on the fourth document image feature data and the fifth document image feature data to obtain feature data of the sixth document image and feature data of the seventh document image; the weighted cross-modal fusion module can be used to perform weighted fusion processing on the sixth document image feature data and the seventh document image feature data to obtain feature data of the first document image.

[0141] Optionally, the acquiring of the document image sample generated based on the target document sample may further include: performing a scale transformation process on the first document image to obtain at least one second document image having a scale different from that of the first document image.

[0142] The using of the initial lightweight target detection model to perform target detection processing on the first document image and the document spectrum map to obtain a prediction result for the image area in the first document image that has the preset quality problem may include: using the initial lightweight target detection model to perform target detection processing on the first document image, the second document image and the document spectrum map to obtain a prediction result for the image area in the first document image that has the preset quality problem.

[0143] Wherein, the multimodal feature extraction model may further include: at least one third feature extraction module, and a scale integration unit in the ZoomNet model; the third feature extraction module and the first feature extraction module may be connected to the scale integration unit, and the scale integration unit may be connected to the frequency-aware cross-modal attention module. Any of the third feature extraction modules may be used to perform feature extraction processing on a second document image to obtain eighth document image feature data. The scale integration unit may be used to perform feature fusion processing on the eighth document image feature data and the fourth document image feature data to obtain ninth document image feature data; the frequency-aware cross-modal attention module may be specifically used to perform feature fusion processing on the ninth document image feature data and the fifth document image feature data to obtain the sixth document image feature data and the seventh document image feature data.

[0144] Optionally, the multimodal feature extraction model may further include: a Mambaformer module; the weighted cross-modal fusion module may be connected to the encoder via the Mambaformer module. The weighted cross-modal fusion module may be specifically used to perform weighted fusion processing on the sixth document image feature data and the seventh document image feature data to obtain the tenth document image feature data. The Mambaformer module may be used to perform feature extraction processing on the tenth document image feature data to obtain the first document image feature data.

[0145] Optionally, the encoder may include multiple encoding layers connected in sequence, the decoder may include multiple decoding layers connected in sequence, at least some of the encoding layers and the decoding layers may include the visual state space module; and the detection head module may include a target number of target detection layers.

[0146] The first encoding layer in the encoder can be connected to the multimodal feature extraction model, the last encoding layer in the encoder can be connected to the first decoding layer in the decoder, and the target number of decoding layers in the decoder can be connected one-to-one with the target number of target detection layers.

[0147] After the first encoding layer of the encoder receives the first certificate image feature data, feature extraction processing can be performed on the first certificate image feature data by using each encoding layer, so that the last encoding layer outputs the second certificate image feature data.

[0148] After the first decoding layer of the decoder receives the second certificate image feature data, each decoding layer may perform feature extraction processing based on the second certificate image feature data to obtain the third certificate image feature data output by each decoding layer.

[0149] The detection head module can perform target detection processing on the received feature data of the third document image by using the target number of target detection layers to obtain a prediction result for the image area in the first document image that has the preset quality problem.

[0150] Optionally, the lightweight object detection model may further include: at least one skip connection module; any of the skip connection modules may be respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship; the encoder, the decoder and the skip connection module may constitute a symmetrical UNet model or an asymmetrical UNet model; wherein the decoding layer connected to any of the skip connection modules may be used to generate the third document image feature data based on the document image feature data output by the previous decoding layer and the document image feature data output by the encoding layer connected to any of the skip connection modules. The decoding layer not connected to the skip connection module may be used to generate the third document image feature data based on the document image feature data output by the previous decoding layer.

[0151] Optionally, the encoder may be an encoder in a Visual Mamba UNet model, and the decoder may be a decoder in the Visual Mamba UNet model.

[0152] because Figure 5 The lightweight target detection model in the embodiment of the method is Figure 2 The structure and working principle of the lightweight target detection model in the embodiment of the method may be consistent, so reference may be made to Figure 2 The working principle of the model in the embodiment of the method is explained to understand Figure 5 The working principle of the lightweight object detection model in is not described in detail here.

[0153] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 6 , is a schematic diagram of the structure of a document image quality detection device provided in an embodiment of this specification. Figure 6 As shown, the document image quality detection device 6 can be implemented as all or part of the electronic device through software, hardware or a combination of both. According to some embodiments, the document image quality detection device 6 may include an acquisition module 61 and a detection module 62, wherein:

[0154] The acquisition module 61 is used to acquire the document image of the user's target document.

[0155] The detection module 62 is used to perform target detection processing on the document image using a lightweight target detection model to obtain a detection result on whether there are preset quality problems in the document image; wherein the lightweight target detection model may include: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder may both include a visual state space module for feature extraction processing.

[0156] Optionally, the acquisition module 61 may include:

[0157] The acquisition unit is used to acquire a first document image obtained by performing image acquisition on the target document.

[0158] The frequency domain transformation unit is used to perform frequency domain transformation processing on the first certificate image to obtain a certificate spectrum diagram.

[0159] The detection module 62 can be specifically used to: use a lightweight target detection model to perform target detection processing on the first document image and the document spectrum diagram to obtain a target detection result that reflects the image area with preset quality problems in the first document image.

[0160] Among them, the lightweight target detection model can also include: a multimodal feature extraction model; the multimodal feature extraction model can be connected to the encoder; the multimodal feature extraction model can be used to perform feature extraction processing and feature fusion processing on the first ID image and the ID spectrum diagram to obtain first ID image feature data; the encoder can be used to perform feature extraction processing on the first ID image feature data to obtain second ID image feature data; the decoder can be used to perform feature extraction processing on the second ID image feature data to obtain third ID image feature data; the detection head module can be used to perform target detection processing on the third ID image feature data to obtain a target detection result for reflecting the image area with preset quality problems in the first ID image.

[0161] Optionally, the multimodal feature extraction model may include: a first feature extraction module, a second feature extraction module, a frequency-aware cross-modal attention module and a weighted cross-modal fusion module; the first feature extraction module and the second feature extraction module may both be connected to the frequency-aware cross-modal attention module, the frequency-aware cross-modal attention module may be connected to the weighted cross-modal fusion module, and the weighted cross-modal fusion module may be connected to the encoder.

[0162] Among them, the first feature extraction module can be used to perform feature extraction processing on the first ID image to obtain fourth ID image feature data; the second feature extraction module can be used to perform feature extraction processing on the ID spectrum diagram to obtain fifth ID image feature data; the frequency-aware cross-modal attention module can be used to perform feature fusion processing on the fourth ID image feature data and the fifth ID image feature data to obtain sixth ID image feature data and seventh ID image feature data; the weighted cross-modal fusion module can be used to perform weighted fusion processing on the sixth ID image feature data and the seventh ID image feature data to obtain first ID image feature data.

[0163] Optionally, the acquisition module may further include: a scale transformation unit, configured to perform scale transformation processing on the first document image to obtain at least one second document image having a scale different from that of the first document image.

[0164] The detection module 62 can be specifically used to: use a lightweight target detection model to perform target detection processing on the first document image, the second document image and the document spectrum diagram, and obtain a target detection result that reflects the image area with preset quality problems in the first document image.

[0165] Among them, the multimodal feature extraction model can also include: at least one third feature extraction module, and a scale integration unit in the ZoomNet model; the third feature extraction module and the first feature extraction module can both be connected to the scale integration unit, and the scale integration unit can be connected to the frequency-aware cross-modal attention module; any of the third feature extraction modules can be used to perform feature extraction processing on a second document image to obtain eighth document image feature data; the scale integration unit can be used to perform feature fusion processing on the eighth document image feature data and the fourth document image feature data to obtain ninth document image feature data; the frequency-aware cross-modal attention module can specifically be used to perform feature fusion processing on the ninth document image feature data and the fifth document image feature data to obtain the sixth document image feature data and the seventh document image feature data.

[0166] Optionally, the multimodal feature extraction model may further include: a Mambaformer module; the weighted cross-modal fusion module may be connected to the encoder through the Mambaformer module; the weighted cross-modal fusion module may be specifically used to perform weighted fusion processing on the sixth ID image feature data and the seventh ID image feature data to obtain tenth ID image feature data; the Mambaformer module may be used to perform feature extraction processing on the tenth ID image feature data to obtain the first ID image feature data.

[0167] Optionally, the encoder may include multiple encoding layers connected in sequence, and the decoder may include multiple decoding layers connected in sequence, and at least some of the encoding layers and the decoding layers may include the visual state space module; and the detection head module may include a target number of target detection layers; the first encoding layer in the encoder may be connected to the multimodal feature extraction model, the last encoding layer in the encoder may be connected to the first decoding layer in the decoder, and the target number of decoding layers in the decoder may be connected one-to-one with the target number of target detection layers.

[0168] After the first encoding layer of the encoder receives the first certificate image feature data, feature extraction processing can be performed on the first certificate image feature data by using each encoding layer, so that the last encoding layer outputs the second certificate image feature data.

[0169] After the first decoding layer of the decoder receives the second certificate image feature data, each decoding layer may perform feature extraction processing based on the second certificate image feature data to obtain the third certificate image feature data output by each decoding layer.

[0170] The detection head module can perform target detection processing on the received third document image feature data by using the target number of target detection layers to obtain a target detection result reflecting an image area with a preset quality problem in the first document image.

[0171] Optionally, the lightweight target detection model may also include: at least one jump connection module; any of the jump connection modules can be respectively connected to a pair of encoding layers and decoding layers having a preset corresponding relationship; the encoder, the decoder and the jump connection module can constitute a symmetric UNet model or an asymmetric UNet model.

[0172] The decoding layer connected to any skip connection module can be used to generate the third document image feature data according to the document image feature data output by the previous decoding layer and the document image feature data output by the encoding layer connected to any skip connection module. The decoding layer not connected to the skip connection module can be used to generate the third document image feature data according to the document image feature data output by the previous decoding layer.

[0173] Optionally, the encoder may be an encoder in a Visual Mamba UNet model, and the decoder may be a decoder in the Visual Mamba UNet model. Figure 6 The device in the embodiment can be applied to a terminal device, and the terminal device can be equipped with the lightweight target detection model.

[0174] Optional, Figure 6The device may further include: a judgment module, configured to judge, based on the detection result, whether the overlap area ratio between the image area with the preset quality problem in the document image and the preset document key area reaches a preset threshold, and obtain a judgment result. An information generation module, configured to generate information for instructing to capture the image of the target document of the user again for document image quality detection if it is determined, based on the judgment result, that the document image has the preset quality problem whose corresponding overlap area ratio reaches a preset threshold.

[0175] Optional, Figure 6 The device may further include: a preprocessing module, configured to preprocess the document image to obtain a preprocessed document image of a target size; the preprocessing includes: at least one of image correction processing and scale change processing. The detection module 62 may be specifically configured to: perform target detection processing on the preprocessed document image using a lightweight target detection model to obtain first position information of an image region reflecting the presence of the preset quality problem in the preprocessed document image.

[0176] The information generation module may include: a second position information acquisition unit, used to acquire second position information of a preset document key area in a preset document template under the target size; the document type to which the preset document template belongs is consistent with the document type to which the target document belongs. A judgment unit, used to judge whether the overlap area ratio between the image area with the preset quality problem in the pre-processed document image and the preset document key area reaches a preset threshold value based on the first position information and the second position information.

[0177] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 7 , is a schematic diagram of a structure of a training device for a lightweight target detection model provided in an embodiment of this specification. Figure 7 As shown, the training device 7 of the lightweight target detection model can be implemented as all or part of the electronic device through software, hardware or a combination of both. According to some embodiments, the training device 7 of the lightweight target detection model may include an acquisition module 71, a determination module 72, and a training module 73, wherein:

[0178] The acquisition module 71 is used to acquire a document image sample generated based on a target document sample.

[0179] The determination module 72 is used to determine the preset label data that the certificate image sample needs to carry; the preset label data is used to reflect the image area in the certificate image sample that has a preset quality problem.

[0180] The training module 73 is used to train the initial lightweight target detection model using the certificate image samples carrying the preset label data to obtain a lightweight target detection model for quality detection of certificate images; wherein the lightweight target detection model may include: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder may both include a visual state space module for feature extraction processing.

[0181] Optionally, the acquisition module 71 may include:

[0182] The acquisition unit is used to acquire a first document image obtained by performing image acquisition on the target document sample.

[0183] The frequency domain transformation unit is used to perform frequency domain transformation processing on the first certificate image to obtain a certificate spectrum diagram.

[0184] The determination module 72 may be specifically configured to determine preset label data for reflecting an image region having a preset quality problem in the first certificate image.

[0185] The training module 73 can be specifically used to: use the initial lightweight target detection model to perform target detection processing on the first document image and the document spectrum diagram to obtain a prediction result for the image area in the first document image that has the preset quality problem; with minimizing the difference between the prediction result and the preset label data as the training goal, adjust the parameters of the initial lightweight target detection model to obtain the lightweight target detection model; wherein the lightweight target detection model can also include: a multimodal feature extraction model; the multimodal feature extraction model is connected to the encoder.

[0186] The multimodal feature extraction model can be used to perform feature extraction and feature fusion processing on the first ID image and the ID spectrum diagram to obtain first ID image feature data; the encoder can be used to perform feature extraction processing on the first ID image feature data to obtain second ID image feature data; the decoder can be used to perform feature extraction processing on the second ID image feature data to obtain third ID image feature data; the detection head module can be used to perform target detection processing on the third ID image feature data to obtain a prediction result for the image area in the first ID image that has the preset quality problem.

[0187] Optionally, the multimodal feature extraction model may include: a first feature extraction module, a second feature extraction module, a frequency-aware cross-modal attention module and a weighted cross-modal fusion module; the first feature extraction module and the second feature extraction module may both be connected to the frequency-aware cross-modal attention module, the frequency-aware cross-modal attention module may be connected to the weighted cross-modal fusion module, and the weighted cross-modal fusion module may be connected to the encoder.

[0188] Among them, the first feature extraction module can be used to perform feature extraction processing on the first ID image to obtain fourth ID image feature data; the second feature extraction module can be used to perform feature extraction processing on the ID spectrum diagram to obtain fifth ID image feature data; the frequency-aware cross-modal attention module can be used to perform feature fusion processing on the fourth ID image feature data and the fifth ID image feature data to obtain sixth ID image feature data and seventh ID image feature data; the weighted cross-modal fusion module can be used to perform weighted fusion processing on the sixth ID image feature data and the seventh ID image feature data to obtain the first ID image feature data.

[0189] Optionally, the acquisition module 71 may further include: a scale transformation unit, configured to perform scale transformation processing on the first certificate image to obtain at least one second certificate image having a scale different from that of the first certificate image.

[0190] The training module 73 can be specifically used to: use the initial lightweight target detection model to perform target detection processing on the first document image, the second document image and the document spectrum diagram, and obtain a prediction result for the image area in the first document image that has the preset quality problem.

[0191] Among them, the multimodal feature extraction model can also include: at least one third feature extraction module, and a scale integration unit in the ZoomNet model; the third feature extraction module and the first feature extraction module can both be connected to the scale integration unit, and the scale integration unit can be connected to the frequency-aware cross-modal attention module; any of the third feature extraction modules can be used to perform feature extraction processing on a second document image to obtain eighth document image feature data; the scale integration unit can be used to perform feature fusion processing on the eighth document image feature data and the fourth document image feature data to obtain ninth document image feature data; the frequency-aware cross-modal attention module can specifically be used to perform feature fusion processing on the ninth document image feature data and the fifth document image feature data to obtain sixth document image feature data and seventh document image feature data.

[0192] Optionally, the multimodal feature extraction model may further include: a Mambaformer module; the weighted cross-modal fusion module may be connected to the encoder through the Mambaformer module; the weighted cross-modal fusion module may be specifically used to perform weighted fusion processing on the sixth ID image feature data and the seventh ID image feature data to obtain tenth ID image feature data; the Mambaformer module may be used to perform feature extraction processing on the tenth ID image feature data to obtain the first ID image feature data.

[0193] Optionally, the encoder may include a plurality of sequentially connected encoding layers, the decoder may include a plurality of sequentially connected decoding layers, at least some of the encoding layers and the decoding layers may include the visual state space module; and the detection head module may include a target number of target detection layers; the first encoding layer in the encoder may be connected to the multimodal feature extraction model, the last encoding layer in the encoder may be connected to the first decoding layer in the decoder, and the target number of decoding layers in the decoder may be connected one-to-one with the target number of target detection layers; after the first encoding layer of the encoder receives the first document image feature data, The first document image feature data can be subjected to feature extraction processing by utilizing each of the encoding layers so that the last encoding layer outputs the second document image feature data; after the first decoding layer of the decoder receives the second document image feature data, feature extraction processing can be performed based on the second document image feature data by utilizing each of the decoding layers so as to obtain the third document image feature data output by each of the decoding layers; the detection head module can perform target detection processing on the received third document image feature data by utilizing the target detection layers of the target number so as to obtain a prediction result for the image area in the first document image that has the preset quality problem.

[0194] Optionally, the lightweight object detection model may further include: at least one skip connection module; any of the skip connection modules may be respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship; the encoder, the decoder and the skip connection module may constitute a symmetrical UNet model or an asymmetrical UNet model; wherein the decoding layer connected to any of the skip connection modules may be used to generate the third document image feature data based on the document image feature data output by the previous decoding layer, and the document image feature data output by the encoding layer connected to any of the skip connection modules. The decoding layer not connected to the skip connection module may be used to generate the third document image feature data based on the document image feature data output by the previous decoding layer. Optionally, the encoder may be an encoder in a visual mamba UNet model, and the decoder may be a decoder in the visual mamba UNet model.

[0195] The present specification also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above Figure 2 and Figure 5 For the method shown, the specific execution process can refer to the specific description of the relevant embodiments of the above method, which will not be repeated here.

[0196] The embodiments of this specification also provide Figure 8 The structural diagram of the electronic device shown in FIG. Figure 8 At the hardware level, the electronic device may include a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 2 and Figure 5 For the method shown, the specific execution process can refer to the specific description of the relevant embodiments of the above method, which will not be repeated here.

[0197] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0198] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for computer-readable storage media, computer program products and Figure 8As for the electronic device shown, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0199] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0200] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0201] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0202] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0203] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0204] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0205] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0206] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0207] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0208] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0209] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0210] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0211] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0212] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0213] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0214] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for detecting the quality of a document image, comprising: Obtain the ID image of the user's target ID; Using a lightweight target detection model, performing target detection processing on the document image to obtain a detection result of whether there is a preset quality problem in the document image; The lightweight object detection model includes: an encoder, a decoder and a detection head module connected in sequence; The encoder and the decoder both include a visual state space module for feature extraction processing.

2. According to the method of claim 1, the step of obtaining the document image of the target document of the user comprises: Acquire a first document image obtained by performing image acquisition on the target document; Performing frequency domain transformation processing on the first certificate image to obtain a certificate spectrum diagram; The method of using a lightweight target detection model to perform target detection processing on the document image to obtain a detection result of whether there is a preset quality problem in the document image includes: Using a lightweight target detection model, target detection processing is performed on the first document image and the document spectrum diagram to obtain a target detection result reflecting an image area having a preset quality problem in the first document image; Wherein, the lightweight target detection model further includes: a multimodal feature extraction model; the multimodal feature extraction model is connected to the encoder; The multimodal feature extraction model is used to perform feature extraction processing and feature fusion processing on the first ID image and the ID spectrum diagram to obtain feature data of the first ID image; The encoder is used to perform feature extraction processing on the first certificate image feature data to obtain second certificate image feature data; The decoder is used to perform feature extraction processing on the second certificate image feature data to obtain third certificate image feature data; The detection head module is used to perform target detection processing on the feature data of the third certificate image to obtain a target detection result for reflecting an image area in the first certificate image where a preset quality problem exists.

3. The method according to claim 2, wherein the multimodal feature extraction model comprises: A first feature extraction module, a second feature extraction module, a frequency-aware cross-modal attention module, and a weighted cross-modal fusion module; The first feature extraction module and the second feature extraction module are both connected to the frequency-aware cross-modal attention module, the frequency-aware cross-modal attention module is connected to the weighted cross-modal fusion module, and the weighted cross-modal fusion module is connected to the encoder; Wherein, the first feature extraction module is used to perform feature extraction processing on the first certificate image to obtain feature data of the fourth certificate image; The second feature extraction module is used to perform feature extraction processing on the document spectrum diagram to obtain fifth document image feature data; The frequency-aware cross-modal attention module is used to perform feature fusion processing on the fourth document image feature data and the fifth document image feature data to obtain the sixth document image feature data and the seventh document image feature data; The weighted cross-modal fusion module is used to perform weighted fusion processing on the sixth certificate image feature data and the seventh certificate image feature data to obtain the first certificate image feature data.

4. The method according to claim 3, wherein obtaining the document image of the target document of the user further comprises: Performing a scale transformation process on the first ID image to obtain at least one second ID image having a scale different from that of the first ID image; The method of using a lightweight target detection model to perform target detection processing on the document image to obtain a detection result of whether there is a preset quality problem in the document image includes: Using a lightweight target detection model, target detection processing is performed on the first document image, the second document image, and the document spectrum diagram to obtain a target detection result reflecting an image area having a preset quality problem in the first document image; The multimodal feature extraction model further includes: at least one third feature extraction module, and a scale integration unit in the ZoomNet model; the third feature extraction module and the first feature extraction module are both connected to the scale integration unit, and the scale integration unit is connected to the frequency-aware cross-modal attention module; Any of the third feature extraction modules is used to perform feature extraction processing on one of the second certificate images to obtain eighth certificate image feature data; The scale integration unit is used to perform feature fusion processing on the eighth certificate image feature data and the fourth certificate image feature data to obtain ninth certificate image feature data; The frequency-aware cross-modal attention module is specifically used to perform feature fusion processing on the ninth document image feature data and the fifth document image feature data to obtain the sixth document image feature data and the seventh document image feature data.

5. According to the method of claim 4, the multimodal feature extraction model further comprises: Mambaformer module; the weighted cross-modal fusion module is connected to the encoder through the Mambaformer module; The weighted cross-modal fusion module is specifically used to perform weighted fusion processing on the sixth certificate image feature data and the seventh certificate image feature data to obtain the tenth certificate image feature data; The Mambaformer module is used to perform feature extraction processing on the tenth certificate image feature data to obtain the first certificate image feature data.

6. The method according to claim 2, wherein the encoder comprises a plurality of sequentially connected encoding layers, the decoder comprises a plurality of sequentially connected decoding layers, and at least some of the encoding layers and the decoding layers comprise the visual state space module; And, the detection head module includes a target number of target detection layers; The first encoding layer in the encoder is connected to the multimodal feature extraction model, the last encoding layer in the encoder is connected to the first decoding layer in the decoder, and the target number of decoding layers in the decoder are connected to the target number of target detection layers in a one-to-one correspondence; After the first encoding layer of the encoder receives the first certificate image feature data, the encoder performs feature extraction processing on the first certificate image feature data by using each encoding layer, so that the last encoding layer outputs the second certificate image feature data; After the first decoding layer of the decoder receives the second certificate image feature data, each decoding layer performs feature extraction processing based on the second certificate image feature data to obtain the third certificate image feature data output by each decoding layer; The detection head module performs target detection processing on the received feature data of the third document image by using the target detection layers of the target number to obtain a target detection result reflecting an image area in the first document image where a preset quality problem exists.

7. According to the method of claim 6, the lightweight object detection model further comprises: at least one skip connection module; Any of the skip connection modules is respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship; The encoder, the decoder and the skip connection module constitute a symmetric UNet model or an asymmetric UNet model; The decoding layer connected to any of the skip connection modules is used to generate the third document image feature data according to the document image feature data output by the previous decoding layer and the document image feature data output by the encoding layer connected to any of the skip connection modules; The decoding layer that is not connected to the skip connection module is used to generate the third document image feature data according to the document image feature data output by the previous decoding layer.

8. According to the method of claim 7, the encoder is an encoder in a Visual Mamba UNet model, and the decoder is a decoder in the Visual Mamba UNet model.

9. The method according to any one of claims 1-8, wherein the method is applied to a terminal device, and the terminal device is equipped with the lightweight target detection model.

10. The method according to any one of claims 1 to 8, wherein after performing target detection processing on the document image using a lightweight target detection model to obtain a detection result on whether there is a preset quality problem in the document image, the method further comprises: Based on the detection result, determine whether the overlap area ratio between the image region with the preset quality problem in the certificate image and the preset certificate key region reaches a preset threshold, and obtain a determination result; If it is determined according to the judgment result that the preset quality problem exists in the document image and the corresponding overlap area ratio reaches the preset threshold, information is generated to instruct the image of the user's target document to be captured again for document image quality detection.

11. The method according to claim 10, further comprising: Preprocessing the document image to obtain a preprocessed document image of a target size; The preprocessing includes: at least one of image correction processing and scale change processing; The method of using a lightweight target detection model to perform target detection processing on the document image to obtain a detection result of whether there is a preset quality problem in the document image includes: Using a lightweight target detection model, performing target detection processing on the pre-processed document image to obtain first position information of an image region reflecting the presence of the preset quality problem in the pre-processed document image; The determining, based on the detection result, whether the overlap area ratio between the image region with the preset quality problem in the certificate image and the preset certificate key region reaches a preset threshold includes: Acquire second position information of a preset certificate key area in a preset certificate template under the target size; the certificate type to which the preset certificate template belongs is consistent with the certificate type to which the target certificate belongs; According to the first position information and the second position information, it is determined whether the overlap area ratio between the image region with the preset quality problem in the pre-processed document image and the preset document key region reaches a preset threshold.

12. A method for training a lightweight target detection model, comprising: Obtain a document image sample generated based on a target document sample; Determining the preset label data that the certificate image sample needs to carry; The preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem; Using the certificate image samples carrying the preset label data to train an initial lightweight object detection model to obtain a lightweight object detection model for quality detection of certificate images; The lightweight target detection model comprises: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder both contain a visual state space module for feature extraction processing.

13. The method according to claim 12, wherein obtaining a document image sample generated based on the target document sample comprises: Acquire a first document image obtained by performing image acquisition on the target document sample; Performing frequency domain transformation processing on the first certificate image to obtain a certificate spectrum diagram; The step of determining the preset label data that the certificate image sample needs to carry includes: Determining preset label data for reflecting an image region having a preset quality problem in the first certificate image; The step of training the initial lightweight object detection model using the document image sample carrying the preset label data to obtain a lightweight object detection model for quality detection of the document image includes: Using an initial lightweight target detection model, target detection processing is performed on the first document image and the document spectrum diagram to obtain a prediction result for an image area in the first document image that has the preset quality problem; Taking minimizing the difference between the prediction result and the preset label data as a training goal, adjusting the parameters of the initial lightweight object detection model to obtain the lightweight object detection model; Wherein, the lightweight target detection model further includes: a multimodal feature extraction model; the multimodal feature extraction model is connected to the encoder; The multimodal feature extraction model is used to perform feature extraction processing and feature fusion processing on the first ID image and the ID spectrum diagram to obtain feature data of the first ID image; The encoder is used to perform feature extraction processing on the first certificate image feature data to obtain second certificate image feature data; The decoder is used to perform feature extraction processing on the second certificate image feature data to obtain third certificate image feature data; The detection head module is used to perform target detection processing on the feature data of the third certificate image to obtain a prediction result for the image area in the first certificate image that has the preset quality problem.

14. The method according to claim 13, wherein the multimodal feature extraction model comprises: A first feature extraction module, a second feature extraction module, a frequency-aware cross-modal attention module, and a weighted cross-modal fusion module; The first feature extraction module and the second feature extraction module are both connected to the frequency-aware cross-modal attention module, the frequency-aware cross-modal attention module is connected to the weighted cross-modal fusion module, and the weighted cross-modal fusion module is connected to the encoder; The first feature extraction module is used to perform feature extraction processing on the first certificate image to obtain feature data of the fourth certificate image; The second feature extraction module is used to perform feature extraction processing on the document spectrum diagram to obtain fifth document image feature data; The frequency-aware cross-modal attention module is used to perform feature fusion processing on the fourth document image feature data and the fifth document image feature data to obtain the sixth document image feature data and the seventh document image feature data; The weighted cross-modal fusion module is used to perform weighted fusion processing on the sixth certificate image feature data and the seventh certificate image feature data to obtain the first certificate image feature data.

15. The method according to claim 14, wherein obtaining the document image sample generated based on the target document sample further comprises: Performing a scale transformation process on the first ID image to obtain at least one second ID image having a scale different from that of the first ID image; The using of the initial lightweight target detection model to perform target detection processing on the first document image and the document spectrum diagram to obtain a prediction result for the image area having the preset quality problem in the first document image includes: Using an initial lightweight target detection model, target detection processing is performed on the first document image, the second document image, and the document spectrum diagram to obtain a prediction result for an image area in the first document image that has the preset quality problem; The multimodal feature extraction model further includes: at least one third feature extraction module, and a scale integration unit in the ZoomNet model; the third feature extraction module and the first feature extraction module are both connected to the scale integration unit, and the scale integration unit is connected to the frequency-aware cross-modal attention module; Any of the third feature extraction modules is used to perform feature extraction processing on one of the second certificate images to obtain eighth certificate image feature data; The scale integration unit is used to perform feature fusion processing on the eighth certificate image feature data and the fourth certificate image feature data to obtain ninth certificate image feature data; The frequency-aware cross-modal attention module is specifically used to perform feature fusion processing on the ninth document image feature data and the fifth document image feature data to obtain the sixth document image feature data and the seventh document image feature data.

16. The method according to claim 15, wherein the multimodal feature extraction model further comprises: Mambaformer module; the weighted cross-modal fusion module is connected to the encoder through the Mambaformer module; The weighted cross-modal fusion module is specifically used to perform weighted fusion processing on the sixth certificate image feature data and the seventh certificate image feature data to obtain the tenth certificate image feature data; The Mambaformer module is used to perform feature extraction processing on the tenth certificate image feature data to obtain the first certificate image feature data.

17. The method according to claim 13, wherein the encoder comprises a plurality of sequentially connected encoding layers, the decoder comprises a plurality of sequentially connected decoding layers, and at least some of the encoding layers and the decoding layers comprise the visual state space module; And, the detection head module includes a target number of target detection layers; The first encoding layer in the encoder is connected to the multimodal feature extraction model, the last encoding layer in the encoder is connected to the first decoding layer in the decoder, and the target number of decoding layers in the decoder are connected to the target number of target detection layers in a one-to-one correspondence; After the first encoding layer of the encoder receives the first certificate image feature data, the encoder performs feature extraction processing on the first certificate image feature data by using each encoding layer, so that the last encoding layer outputs the second certificate image feature data; After the first decoding layer of the decoder receives the second certificate image feature data, each decoding layer performs feature extraction processing based on the second certificate image feature data to obtain the third certificate image feature data output by each decoding layer; The detection head module performs target detection processing on the received feature data of the third document image by using the target detection layers of the target number to obtain a prediction result for the image area in the first document image that has the preset quality problem.

18. The method according to claim 17, wherein the lightweight object detection model further comprises: at least one skip connection module; Any of the skip connection modules is respectively connected to a pair of the encoding layer and the decoding layer having a preset corresponding relationship; The encoder, the decoder and the skip connection module constitute a symmetric UNet model or an asymmetric UNet model; The decoding layer connected to any of the skip connection modules is used to generate the third document image feature data according to the document image feature data output by the previous decoding layer and the document image feature data output by the encoding layer connected to any of the skip connection modules; The decoding layer that is not connected to the skip connection module is used to generate the third document image feature data according to the document image feature data output by the previous decoding layer.

19. The method according to claim 18, wherein the encoder is an encoder in a Visual Mamba UNet model, and the decoder is a decoder in the Visual Mamba UNet model.

20. A document image quality detection device, comprising: An acquisition module, used to acquire a certificate image of a user's target certificate; A detection module, used to perform target detection processing on the document image using a lightweight target detection model to obtain a detection result on whether there is a preset quality problem in the document image; The lightweight target detection model comprises: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder both contain a visual state space module for feature extraction processing.

21. A training device for a lightweight target detection model, comprising: An acquisition module, used to acquire a document image sample generated based on a target document sample; A determination module, used to determine the preset label data that the certificate image sample needs to carry; The preset label data is used to reflect the image area of ​​the certificate image sample that has a preset quality problem; A training module, used to train an initial lightweight object detection model using the document image samples carrying the preset label data, to obtain a lightweight object detection model for quality detection of the document image; The lightweight target detection model comprises: an encoder, a decoder and a detection head module connected in sequence; the encoder and the decoder both contain a visual state space module for feature extraction processing.

22. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 19.

23. An electronic device, comprising: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method as claimed in any one of claims 1 to 19.

Images