A method and system for incoming part quality inspection based on a multi-layer composite model

Abstract

This invention relates to an incoming material quality inspection method and system based on a multi-layer composite model. The method sequentially uses a multi-layer model to perform a logical quality inspection process on incoming materials and provides feedback on the inspection results. Specifically, it can perform quality inspection on ID cards based on an end-to-end image classification network using spatial and noise features. Noise features can effectively help the network identify whether the image has been tampered with. Simultaneously, the end-to-end image classification network can effectively avoid the accumulation of errors caused by too many modules performing quality inspections and reduce engineering complexity. Furthermore, this invention also uses a multi-layer model to perform group photo detection on other incoming material images, thereby detecting whether the incoming material image data contains group photos of account managers and customers, thus adapting to incoming material quality inspection requirements.

Description

Technical Field

[0001] This invention relates to the fields of financial information security management and deep learning technology, and in particular to an incoming document quality inspection method and system based on a multi-layer composite model. Background Technology

[0002] Traditional risk management methods typically rely on manual review of application materials to ensure their legality and compliance. This approach struggles to handle the surge in applications, leading to resource strain and reduced processing efficiency. Furthermore, manual review is overly dependent on the individual experience of reviewers, increasing the risk of oversights.

[0003] To address this issue, existing technologies also utilize computer-aided processing methods to assist auditors in reviewing incoming documents. For example, patent application number 202010357043, entitled "Automatic Method and System for Checking Document Image Quality," discloses an automatic method for checking document image quality. First, the document area is located within the document image; then, the document area image is corrected using perspective transformation; the recognizability of the corrected document image is determined; if recognizable, the recognized content is output and the next step is performed; if not recognizable, the quality check result is directly output; based on different quality check indicators and their true / false values, the corrected document image is classified according to these indicators, and a binary classification result is output; finally, the document image quality check result is output. This method can automate the review of document image quality, particularly assisting in determining whether document images contain risks such as forgery or alteration, and can be used to assist auditors in reviewing incoming materials.

[0004] However, while the aforementioned methods can be used to detect the quality of ID card images, they are essentially multi-stage sequential detection methods. The final detection result is highly dependent on the accuracy of each different detection stage, making it susceptible to interference from the accuracy of a particular stage, thus affecting the overall detection model's performance. Since the overall method involves a series of processing steps, including ID card region localization, perspective transformation of the ID card image, and binary classification of the corrected ID card image quality indicators, deviations in previous steps can affect subsequent steps, leading to error accumulation. Therefore, the final detection result is easily affected by the accuracy of a particular step. Furthermore, the specific training objectives for each step differ, making it easy for the trained model to achieve local optima in individual modules, while the overall performance struggles to reach its optimal level. In addition, operations such as affine transformations in these methods may cause the image to lose some original information, and this binary classification of ID card images based solely on spatial features is easily misled by tampered images, resulting in incorrect classification results.

[0005] Therefore, it can be seen that in the actual use of the existing document image quality inspection model, many factors may affect the accuracy of the quality inspection model, thereby affecting the efficiency and security of online document review. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes an incoming document quality inspection method and system based on a multi-layer composite model. An end-to-end image classification network using spatial and noise features is used to inspect ID cards. Noise features effectively help the network identify whether the image has been tampered with. Simultaneously, the end-to-end image classification network effectively avoids the accumulation of errors caused by too many modules performing quality inspections and reduces engineering complexity. Furthermore, this invention also uses a multi-layer model to perform group photo detection on other incoming document images, thereby detecting whether the incoming document image data contains group photos of account managers and customers, thus adapting to incoming document quality inspection requirements.

[0007] To achieve the above objectives, the technical solution adopted by the present invention includes:

[0008] An incoming part quality inspection method based on a multi-layer composite model, characterized by comprising:

[0009] S1. Obtain the incoming image set, use the first model to determine whether the incoming image set includes both ID card images and group photos. If it is determined that the incoming image set does not include both ID card images and group photos, stop the quality inspection process and output the first quality inspection failure information.

[0010] S2. When it is determined that the incoming image set includes both document images and group photo images, the second model is further used to determine whether the document images meet the preset quality requirements. When it is determined that the document images do not meet the preset quality requirements, the quality inspection process is stopped and the second quality inspection non-compliance information is output.

[0011] S3. When it is determined that the incoming image set includes both ID card images and group photos, the third model is further used to determine whether the group photo image can match the preset specified person's facial information. When it is determined that the group photo image cannot match the preset specified person's facial information, the quality inspection process is stopped and the third quality inspection failure information is output.

[0012] S4. When it is determined that the group photo image can match the preset facial information of the designated person, further identify the facial information of the user in the group photo image other than the facial information of the designated person.

[0013] S5. When it is determined that the document image meets the preset quality requirements and the group photo image can match the preset facial information of the designated person, further determine whether the user's facial information matches the document image. When it is determined that the user's facial information does not match the document image, stop the quality inspection process and output the fourth quality inspection failure information.

[0014] S6. When it is determined that the user's facial information matches the ID card image, output the quality inspection qualified information.

[0015] Furthermore, step S2 also includes:

[0016] Perform format verification on the acquired document image, convert the document image into an RGB three-channel image, and adjust it to the preset size.

[0017] Furthermore, the second model includes an end-to-end image classification network model based on spatial and noise features.

[0018] Furthermore, the step of using the second model to determine whether the document image meets the preset quality requirements includes:

[0019] Constrained convolutional layers are used to perform dynamic preprocessing on the document images, and convolutional neural networks are used to learn noise features to obtain the corresponding noisy images.

[0020] The global attention weighting module is used to perform attention weighting processing on the document image and the noisy image to obtain RGB features and noise features;

[0021] The RGB features and noise features are concatenated and input into a ResNet network to obtain the output features;

[0022] The multi-scale feature fusion layer is used to combine the output features, and then the classification layer is used to determine whether the document image meets the preset quality requirements.

[0023] Furthermore, the first model includes a specified image feature detection model; the third model includes a face feature comparison model.

[0024] Further, step S4 includes:

[0025] Identify the coordinates and number of faces in a group photo image.

[0026] This invention also relates to an incoming parts quality inspection system based on a multi-layer composite model, characterized in that it includes:

[0027] The incoming document processing module is used to determine, using the first model, whether the incoming document image set includes both an ID image and a group photo image;

[0028] The document image quality inspection module is used to determine whether the document image meets the preset quality requirements using a second model.

[0029] The group photo image quality inspection module is used to determine whether a group photo image can match the preset facial information of designated individuals using a third model.

[0030] The face recognition module is used to identify user face information in group photos, in addition to the face information of designated persons.

[0031] The joint quality inspection module is used to determine whether the user's facial information matches the image on the identification document.

[0032] The present invention also relates to a computer-readable storage medium, characterized in that the storage medium stores a computer program, which, when executed by a processor, implements the above-described method.

[0033] The present invention also relates to an electronic device, characterized in that it includes a processor and a memory;

[0034] The memory is used to store the set of incoming images and the facial information of designated personnel;

[0035] The processor is used to execute the above method by calling the incoming image set and the facial information of the specified person.

[0036] The present invention also relates to a computer program product, including a computer program and / or instructions, characterized in that the computer program and / or instructions, when executed by a processor, implement the steps of the above-described method.

[0037] The beneficial effects of this invention are as follows:

[0038] The incoming document quality inspection method and system based on a multi-layer composite model described in this invention uses an end-to-end image classification network based on spatial and noise features to inspect ID cards. Noise features effectively help the network identify whether the image has been tampered with, providing the model with more dimensional original information. Simultaneously, a multi-scale feature fusion module combines deep and shallow features, improving the model's accuracy in classifying different quality inspection indicators for ID card images and reducing the false detection rate of tampered images. Furthermore, the end-to-end image classification network enhances the richness of ID card image features compared to existing technologies and reduces the cumulative error between multiple modules through an end-to-end approach, thereby effectively improving the accuracy of ID card image quality inspection classification and reducing engineering complexity. In addition, this invention also uses a multi-layer model to perform group photo detection on other incoming document images, thereby detecting whether the incoming document image data contains group photos of account managers and customers, adapting to incoming document quality inspection requirements. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the incoming part quality inspection method based on a multi-layer composite model according to the present invention.

[0040] Figure 2 This is a schematic diagram of the incoming part quality inspection system based on a multi-layer composite model according to the present invention. Detailed Implementation

[0041] To better understand the content of this invention, a detailed description will be provided in conjunction with the accompanying drawings and embodiments.

[0042] The first aspect of this invention relates to a process flow as follows: Figure 1 The incoming part quality inspection method based on a multi-layer composite model shown includes:

[0043] S1. Obtain the incoming image set. Use the first model to determine whether the incoming image set includes both an ID image and a group photo. If the incoming image set does not include both an ID image and a group photo, stop the quality inspection process and output the first quality inspection failure information.

[0044] The first model is preferably a specified image feature detection model, which is used to initially detect whether the incoming image set contains the necessary document images and group photos, but does not need to judge the specific image quality.

[0045] S2. When it is determined that the incoming image set includes both document images and group photos, the second model is used to determine whether the document images meet the preset quality requirements. When it is determined that the document images do not meet the preset quality requirements, the quality inspection process is stopped and the second quality inspection non-compliance information is output.

[0046] Preferably, the document image should be preprocessed, including performing format verification on the acquired document image, converting the document image into an RGB three-channel image, and adjusting it to a preset size.

[0047] For example, the input document image may be in PNG format, and PNG format images are four-channel, so it is necessary to first convert the input document image into an RGB three-channel image, and then perform a resize operation on the document image to scale it to a size of 256×256.

[0048] The second model includes an end-to-end image classification network model based on spatial and noise features. Specifically, after obtaining the preprocessed document image, a constrained convolutional layer (or a suitable SRM filter) is first used to perform dynamic preprocessing on the preprocessed image, enabling the convolutional neural network to adaptively learn noise features to obtain a noisy image. Then, the RGB image and the noisy image are respectively passed through a global attention weighting module to obtain attention-enhanced RGB features and noise features. The two features are then concatenated and input into a ResNet network. A multi-scale feature fusion layer combines the features output by different ResBlocks. Finally, a classification layer classifies the input document image into a reproduced image, a normal image, an image with missing corners, an occluded image, and a blurred image.

[0049] S3. When it is determined that the incoming image set includes both ID card images and group photos, the third model is further used to determine whether the group photo image can match the preset designated person's facial information. When it is determined that the group photo image cannot match the preset designated person's facial information, the quality inspection process is stopped and the third quality inspection failure information is output.

[0050] Preferably, the third model includes a facial feature comparison model, which is used to compare and match the faces identified in the group photo with the pre-set facial information of designated personnel, such as comparing with the pre-stored facial information of the account manager responsible for handling business.

[0051] In actual execution, steps S2 and S3 can be executed sequentially or synchronously and independently.

[0052] S4. When it is determined that the group photo image can match the preset facial information of the designated person, further identify the facial information of the user in the group photo image other than the facial information of the designated person, including identifying the facial coordinates and number of faces in the group photo image.

[0053] Specifically, the input image can be preprocessed with format validation and image denoising, and then the preprocessed image can be input into the backbone network to extract image features. Then, a multi-scale feature fusion layer can be used to combine high-level semantic information and shallow semantic information to retain more original feature information, thereby generating multi-scale features. Finally, the multi-scale features are input into the prediction layer to obtain the face coordinates and number of faces in the input image.

[0054] The obtained user facial information should be able to be matched with the object's face provided in the document image to ensure the compliance of the submitted documents.

[0055] S5. When it is determined that the document image meets the preset quality requirements and the group photo image can match the preset facial information of the designated person, further determine whether the user's facial information matches the document image. When it is determined that the user's facial information does not match the document image, stop the quality inspection process and output the fourth quality inspection failure information.

[0056] S6. When it is determined that the user's facial information matches the ID card image, output the quality inspection qualified information.

[0057] Another aspect of this invention relates to an incoming part quality inspection system based on a multi-layer composite model, the structure of which is as follows: Figure 2 As shown, it includes:

[0058] The incoming document processing module is used to determine, using the first model, whether the incoming document image set includes both an ID image and a group photo image;

[0059] The document image quality inspection module is used to determine whether the document image meets the preset quality requirements using a second model.

[0060] The group photo image quality inspection module is used to determine whether a group photo image can match the preset facial information of designated individuals using a third model.

[0061] The face recognition module is used to identify user face information in group photos, in addition to the face information of designated persons.

[0062] The joint quality inspection module is used to determine whether the user's facial information matches the image on the identification document.

[0063] By using this system, the aforementioned computational processing methods can be executed and the corresponding technical effects can be achieved.

[0064] Embodiments of the present invention also provide a computer-readable storage medium capable of implementing all the steps of the methods in the above embodiments, wherein the computer-readable storage medium stores a computer program that, when executed by a processor, implements all the steps of the methods in the above embodiments.

[0065] Embodiments of the present invention also provide an electronic device for performing the above-described method. As an implementation device for the method, the electronic device has at least a processor and a memory. In particular, the memory stores data and related computer programs required for performing the method, such as a set of incoming images and facial information of designated personnel. The processor calls the data and programs in the memory to execute all the steps of the method and obtain the corresponding technical effect.

[0066] Preferably, the electronic device may include a bus architecture, which may include any number of interconnected buses and bridges. The bus will include various circuits linked together by one or more processors and memories. The bus may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface between the bus and the receiver and transmitter. The receiver and transmitter may be the same element, i.e., a transceiver, providing a unit for communicating with various other systems over a transmission medium. The processor is responsible for managing the bus and general processing, while the memory may be used to store data used by the processor during operation.

[0067] Additionally, the electronic device may further include components such as a communication module, an input unit, an audio processor, a display, and a power supply. The processor (or controller, operating control) used may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of the electronic device; the memory may be one or more of a buffer, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices, which can store the aforementioned data information, and may also store programs for executing the information, and the processor can execute the program stored in the memory to achieve information storage or processing, etc.; the input unit is used to provide input to the processor, for example, it can be a button or touch input device; the power supply is used to provide power to the electronic device; the display is used to display images and text, for example, it can be an LCD display. The communication module is a transmitter / receiver that transmits and receives signals via an antenna. The communication module (transmitter / receiver) is coupled to the processor to provide input signals and receive output signals, which can be the same as in conventional mobile communication terminals. Based on different communication technologies, multiple communication modules can be incorporated into the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) is also coupled to a speaker and microphone via an audio processor to provide audio output through the speaker and receive audio input from the microphone, thereby enabling typical telecommunications functions. The audio processor can include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor is coupled to a central processing unit, enabling on-device recording via the microphone and on-device playback of stored sound via the speaker.

[0068] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0070] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0072] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An incoming part quality inspection method based on a multi-layer composite model, characterized in that, include: S1. Obtain the incoming image set, use the first model to determine whether the incoming image set includes both ID card images and group photos. If it is determined that the incoming image set does not include both ID card images and group photos, stop the quality inspection process and output the first quality inspection failure information. S2. When it is determined that the incoming image set includes both document images and group photo images, the second model is further used to determine whether the document images meet the preset quality requirements. When it is determined that the document images do not meet the preset quality requirements, the quality inspection process is stopped and the second quality inspection non-compliance information is output. S3. When it is determined that the incoming image set includes both ID card images and group photos, the third model is further used to determine whether the group photo image can match the preset specified person's facial information. When it is determined that the group photo image cannot match the preset specified person's facial information, the quality inspection process is stopped and the third quality inspection failure information is output. S4. When it is determined that the group photo image can match the preset facial information of the designated person, further identify the facial information of the user in the group photo image other than the facial information of the designated person. S5. When it is determined that the document image meets the preset quality requirements and the group photo image can match the preset facial information of the designated person, further determine whether the user's facial information matches the document image. When it is determined that the user's facial information does not match the document image, stop the quality inspection process and output the fourth quality inspection failure information. S6. When it is determined that the user's facial information matches the ID card image, output the quality inspection qualified information; Step S2 further includes: Perform format verification on the acquired document image, convert the document image into an RGB three-channel image, and adjust it to a preset size; The second model includes an end-to-end image classification network model based on spatial and noise features; The step of using the second model to determine whether the document image meets the preset quality requirements includes: Constrained convolutional layers are used to perform dynamic preprocessing on the document images, and convolutional neural networks are used to learn noise features to obtain the corresponding noisy images. The global attention weighting module is used to perform attention weighting processing on the document image and the noisy image to obtain RGB features and noise features; The RGB features and noise features are concatenated and input into a ResNet network to obtain the output features; The multi-scale feature fusion layer is used to combine the output features, and then the classification layer is used to determine whether the document image meets the preset quality requirements.

2. The method as described in claim 1, characterized in that, The first model includes a specified image feature detection model; the third model includes a face feature comparison model.

3. The method as described in claim 1, characterized in that, Step S4 includes: Identify the coordinates and number of faces in a group photo image.

4. An incoming parts quality inspection system based on a multi-layer composite model, characterized in that, The system is used to perform any one of the methods of claims 1-3, comprising: The incoming document processing module is used to determine, using the first model, whether the incoming document image set includes both an ID image and a group photo image; The document image quality inspection module is used to determine whether the document image meets the preset quality requirements using a second model. The group photo image quality inspection module is used to determine whether a group photo image can match the preset facial information of designated individuals using a third model. The face recognition module is used to identify user face information in group photos, in addition to the face information of designated persons. The joint quality inspection module is used to determine whether the user's facial information matches the image on the identification document.

5. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method of any one of claims 1 to 3.

6. An electronic device, characterized in that, Including processor and memory; The memory is used to store the set of incoming images and the facial information of designated personnel; The processor is configured to execute the method of any one of claims 1 to 3 by invoking the input image set and the facial information of a specified person.

7. A computer program product, comprising a computer program and / or instructions, characterized in that, When the computer program and / or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 3.

Images