Intelligent certificate management system and method for multi-type entry and exit certificates based on image processing

Abstract

The invention provides an intelligent certificate management system and method for multi-type entry and exit certificates based on image processing, and relates to the field of image processing and certificate management. The identity recognition module is used for carrying out identity verification based on a certificate storage request or a certificate taking-out request initiated by a user; the certificate access cabinet body is used for receiving a certificate placed by a user in response to a certificate storage request initiated by the user after the identity verification is passed; the certificate analysis module is used for responding to a certificate storage request initiated by a user and carrying out anomaly detection and information identification on a certificate placed by the user; the certificate storing and taking cabinet body is used for storing certificates based on the abnormal detection and information identification results of the certificate analysis module; and the certificate access cabinet body is also used for taking out the stored certificates in response to a certificate taking-out request initiated by the user after the identity verification is passed, and has the advantage of realizing automatic management of the certificates.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and certificate management, and in particular to an intelligent certificate management system and method for multiple types of entry and exit certificates based on image processing. BACKGROUND

[0002] In the current situation of increasing cross-border exchanges and continuous digitalization, the demand for full-life-cycle controllable management of certificates for staff members is becoming increasingly prominent. However, the existing certificate management equipment has many shortcomings.

[0003] Most intelligent cabinets have poor compatibility and can only support standard-thickness passports or single card-type certificates. Diversified certificates such as ultra-thin passports, old passports, passport stickers, passports with paper clips, and passports with adhesive protective sleeves cannot be adapted. When facing these different types of certificates, manual classification and storage are required, resulting in low management efficiency. The existing equipment has relatively single functions and is mainly focused on basic functions such as certificate storage and retrieval. It lacks core management functions such as electronic registration of loaning and automatic detection of unqualified certificates. This makes it necessary to perform additional manual recording during the certificate management process, which not only consumes manpower but also easily leads to registration errors, omissions, and expiration forgetfulness.

[0004] Therefore, it is necessary to provide an intelligent certificate management system and method for multiple types of entry and exit certificates based on image processing to realize the automatic management of certificates. SUMMARY

[0005] The present application provides an intelligent certificate management system for multiple types of entry and exit certificates based on image processing, which includes an interactive module, a certificate storage and retrieval cabinet, and an identity recognition module and a certificate analysis module arranged in the certificate storage and retrieval cabinet. The interactive module is used to receive a certificate storage request or a certificate retrieval request initiated by a user. The identity recognition module is used to perform identity verification based on the certificate storage request or the certificate retrieval request initiated by the user. The certificate storage and retrieval cabinet is used to receive a certificate placed by the user in response to the certificate storage request initiated by the user after the identity verification is passed. The certificate analysis module is used to perform abnormality detection and information recognition on the certificate placed by the user in response to the certificate storage request initiated by the user. The certificate storage and retrieval cabinet is used to store the certificate based on the results of the abnormality detection and information recognition by the certificate analysis module. The certificate storage and retrieval cabinet is also used to retrieve the stored certificate in response to the certificate retrieval request initiated by the user after the identity verification is passed.

[0006] Further, the certificate analysis module is further configured to: perform tilt detection on the certificate placed by the user; after the tilt detection passes, perform quantity detection on the certificate placed by the user based on the certificate storage request initiated by the user; after the quantity detection passes, perform type matching detection on the certificate placed by the user based on the certificate storage request initiated by the user; after the type matching detection passes, perform page-turning outward detection on the certificate placed by the user; after the page-turning outward detection passes, perform stacking standard detection on the certificate placed by the user; and after the stacking standard detection passes, perform physical state detection on the certificate placed by the user based on the certificate storage request initiated by the user.

[0007] Further, the certificate analysis module is further configured to: generate a contour binary image of the certificate placed by the user through an infrared array sensor; determine four corner points of the certificate placed by the user based on the contour binary image of the certificate placed by the user; and perform tilt detection on the certificate placed by the user based on the four corner points of the certificate placed by the user.

[0008] Further, the certificate analysis module is further configured to: acquire an image of the certificate placed by the user; determine the type of the certificate placed by the user based on the image of the certificate placed by the user, wherein the type of the certificate placed by the user is a card type or a book type; and perform type matching detection on the certificate placed by the user based on the certificate storage request initiated by the user and the type of the certificate placed by the user.

[0009] Further, the certificate analysis module is further configured to: acquire an image of the certificate placed by the user; perform preprocessing on the image of the certificate placed by the user and segmentation to determine a region image of each certificate placed by the user; and for each certificate placed by the user, perform page-turning outward detection based on the region image of the certificate, the straightness, the inner page protrusion, and the thickness uniformity of the certificate.

[0010] Further, the certificate analysis module is further configured to: acquire an image of the certificate placed by the user; perform preprocessing on the image of the certificate placed by the user to generate a preprocessed image; perform edge feature enhancement, double-threshold fusion, and morphological post-processing on the preprocessed image to generate a twice-processed image; and convert the twice-processed image to an HSV color space, count a pixel ratio, and perform stripe distribution feature detection to perform page-turning outward detection.

[0011] Further, the certificate analysis module is further configured to: acquire an image of the certificate placed by the user and perform preprocessing to generate a preprocessed image; perform edge detection and contour extraction on the preprocessed image, and perform reference line fitting and deviation calculation for stacking standard detection.

[0012] Further, the certificate analysis module is further used for: collecting an image of the certificate placed by the user, performing damage, stain and foreign matter detection; performing chip detection on the certificate placed by the user; and performing information matching detection on the certificate placed by the user based on a certificate storage request initiated by the user.

[0013] Further, the certificate access cabinet body comprises a certificate access unit, a self-adaptive storage unit, an automatic transmission unit, a local certificate page turning unit and an automatic inventory unit, wherein the self-adaptive storage unit comprises a plurality of self-adaptive certificate storage compartments, the size of the self-adaptive certificate storage compartment is adjustable, the automatic transmission unit is used for transmitting the certificate from the certificate access unit to the self-adaptive storage unit or transmitting the certificate from the self-adaptive storage unit to the certificate access unit, the local certificate page turning unit is used for turning the pages of the local certificate, and the automatic inventory unit is used for automatically inventorying the stored certificate.

[0014] The application provides an intelligent certificate management method for multiple types of entry and exit certificates based on image processing.

[0015] Compared with the prior art, the intelligent certificate management system and method for multiple types of entry and exit certificates based on image processing provided by the application have at least the following beneficial effects: 1. The certificate placed by the user is detected in multiple dimensions. From the inclination detection to ensure that the certificate is placed upright, to the quantity detection to avoid overplacement or underplacement; from the type matching detection to ensure that the certificate type is correct, to the page turning outward and the stacked placement standard detection to maintain the good state of the certificate, and then to the physical state detection to check the damage, stain and other problems. The contour binary image is generated by the infrared array sensor to assist the inclination detection, and the image processing technology is used for other detection, so that the abnormal situation can be accurately recognized, the compliance of the stored certificate is effectively guaranteed, the subsequent use and management are avoided due to the certificate problem, and reliable basic guarantee is provided for the certificate management.

[0016] 2. The certificate access cabinet body is designed scientifically and reasonably, and comprises a certificate access unit, a self-adaptive storage unit and the like. The self-adaptive storage unit comprises multiple size-adjustable self-adaptive certificate storage compartments, which can meet the storage requirements of different types of certificates; the automatic transmission unit realizes the rapid transmission of the certificate between units, improves the access efficiency; the local certificate page turning unit facilitates the operation of the local certificate; and the automatic inventory unit can grasp the certificate storage situation at any time. Intelligent access management is realized, the user only needs to initiate a request, and the identity verification, certificate detection, access and the like can be automatically completed, so that the convenience and smoothness of operation are greatly improved.

[0017] 3. The image processing-based page turning outward detection method greatly improves the accuracy and reliability of entry and exit certificate management. The first method collects images and pre-processes segmentation to determine the image of each certificate area, analyzes straightness, inner page protrusion, and thickness uniformity, and accurately judges the certificate page turning situation from multiple physical characteristics to avoid misjudgment. The second method further detects based on color and stripe information by performing edge enhancement, double threshold fusion, and morphological processing on the pre-processed image and converting it to HSV color space to count features. The combination of the two methods can comprehensively and meticulously detect the certificate page turning outward state, ensure that the certificates stored in the cabinet meet the specifications, provide a solid guarantee for subsequent certificate management and use, and reduce various risks caused by improper placement of certificates. BRIEF DESCRIPTION OF DRAWINGS

[0018] The present specification will be further illustrated in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbers represent the same structures, wherein: Figure 1 is a module schematic diagram of an intelligent certificate management system for multiple types of entry and exit certificates based on image processing according to some embodiments of the present specification; Figure 2 is a structural schematic diagram of a certificate access cabinet according to some embodiments of the present specification; Figure 3 is a flowchart of abnormality detection and information recognition of a certificate placed by a user according to some embodiments of the present specification; Figure 4 is a schematic diagram of the inclination angle of a certificate according to some embodiments of the present specification. DETAILED DESCRIPTION

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some examples or embodiments of the present specification, and for those skilled in the art, the present specification can be applied to other similar scenarios without creative labor. Unless it is obvious from the language environment or otherwise stated, the same reference numbers in the drawings represent the same structure or operation.

[0020] Figure 1 is a module schematic diagram of an intelligent certificate management system for multiple types of entry and exit certificates based on image processing according to some embodiments of the present specification, such as Figure 1As shown, the intelligent certificate management system based on image processing for multiple types of entry and exit certificates can include an interactive module, a certificate access cabinet, and an identity recognition module and a certificate analysis module arranged in the certificate access cabinet.

[0021] In some embodiments, the certificate access cabinet includes a certificate access unit, an adaptive storage unit, an automatic transmission unit, a book-type certificate page turning unit, and an automatic inventory unit. The adaptive storage unit includes a plurality of adaptive storage compartments, the size of which is adjustable. The automatic transmission unit is used to transmit the certificate from the certificate access unit to the adaptive storage unit or from the adaptive storage unit to the certificate access unit. The book-type certificate page turning unit is used to turn the pages of the book-type certificate. The automatic inventory unit is used to automatically inventory the stored certificates.

[0022] Specifically, the adaptive storage unit includes a plurality of adaptive storage compartments with adjustable sizes, which greatly improves its adaptability to different types of certificates. The width of each storage compartment can be adjusted in the range of 50-120mm, and the height can be adjusted in the range of 30-100mm, which can accommodate certificates with a thickness of not more than 80mm. This adjustment function is achieved by a stepper motor drive, ensuring accurate and stable adjustment.

[0023] For special-shaped certificates such as certificates with rounded corners or cut corners, flexible silicone buffer strips are specially added to the inside of the storage compartment, with a thickness of 5mm. This design effectively prevents the edges of the certificate from being damaged during storage, providing a safe storage environment for various special-shaped certificates. Whether it is a pure card-type certificate (such as a Hong Kong and Macao pass card), a book-type certificate (such as a thick and thin passport), or a card and book combined certificate, this adaptive storage compartment can perfectly adapt to meet the diverse storage needs of certificates.

[0024] The automatic transmission unit is a key link for the circulation of certificates within the cabinet. Its transmission mechanism is composed of a synchronous belt (e.g., polyurethane material, width 20mm) and a linear guide rail, and is equipped with a servo motor. This combination design allows the transmission speed to be adjusted in the range of 50-150mm / s, with a transmission accuracy of ±0.1mm, which can smoothly transmit certificates of different thicknesses (2-50mm) without damage and with accurate positioning.

[0025] To prevent the certificate from being stuck during transmission, multiple sets of infrared opposite sensors are arranged in the transmission channel, with a distance of 10mm between the infrared opposite sensors. These infrared opposite sensors can detect the transmission state of the certificate in real time. Once a stuck condition is detected, the transmission mechanism is automatically controlled to run in reverse for adjustment. If the number of stuck times reaches or exceeds 3 times, an alarm mechanism will be triggered, and the administrator will be notified in time to handle it, effectively avoiding equipment failure and certificate damage caused by sticking.

[0026] The formula certificate flipping unit is integrated in the certificate storage compartment of the intelligent certificate cabinet, has a complex structure and powerful functions, and mainly consists of a bearing positioning assembly, a composite flipping assembly, a visual recognition assembly, and a main controller.

[0027] The bearing positioning assembly includes a certificate accommodating groove, an elastic side stopper, and a spine limiting block. The bottom of the certificate accommodating groove is provided with a pressure sensor, which can accurately identify the thickness of the certificate and provide important parameters for subsequent flipping operations. The elastic side stopper is linked with the main controller through an electromagnetic lock to realize clamping positioning of the certificate and ensure the stability of the certificate during the flipping process. The spine limiting block can be lifted in the vertical direction and can adapt to certificates with different binding thicknesses, further improving the universality of the flipping unit.

[0028] The composite flipping assembly is arranged at the front end of the accommodating groove and is the core part of the formula certificate flipping. The vacuum chuck assembly includes an array of three micro-suction cups, and the suction cup negative pressure is adjusted through a PWM waveform, which can adapt to certificates of different materials such as paper and plastic sealing. The suction cup seat is moved in X / Y / Z directions through a three-axis sliding table to ensure that the suction cup can accurately reach the specified position of the certificate page. The auxiliary page turning assembly is provided with a rotatable brush roller on one side of the suction cup, and the rotation speed of the brush roller is matched with the moving speed of the suction cup for separating the adhered pages. A low-power ultrasonic generator is additionally arranged beside the brush roller. If the visual recognition detects that the page separation gap is ≤0.5 mm during flipping, it is determined that the pages are severely adhered, and the ultrasonic wave will be started to cooperate with the brush roller for separation. If single separation fails, the system will automatically adjust the friction wheel pressure (for example, increase by 0.05 N each time) and the suction cup negative pressure value (for example, increase by 0.01 MPa each time), and at most 3 attempts will be made. If it still fails, it is marked as “flipping abnormal” and a warning is pushed, ensuring the accuracy and reliability of the flipping operation. The flattening mechanism includes a transparent pressure cover plate and a micro-cylinder, and the pressure cover plate is rotated through a hinge shaft. After flipping, the cylinder drives the pressure cover plate to press down, so that the pages are attached to the bottom surface of the accommodating groove, ensuring the flatness of the pages and facilitating subsequent identification and storage.

[0029] The visual recognition assembly includes a high-definition camera and a fill light. The camera is arranged directly above the accommodating groove and is used to collect page images and identify page numbers and information areas. The fill light uses a ring-shaped LED, which can automatically adjust the color temperature according to the bottom color of the certificate to provide a good shooting environment for the camera, ensuring clear and accurate images and improving recognition accuracy.

[0030] The main controller includes an MCU processor, a driving circuit, and a communication module. The MCU is connected with the main controller of the certificate cabinet through a CAN bus, receives storage and access certificate / inventory instructions, and outputs control signals to each execution unit to realize precise control and coordination of the entire flipping process.

[0031] The automatic inventory unit provides an efficient and accurate inventory method for certificate management, supporting different types of inventory operations for card-type certificates and book-type certificates.

[0032] For card-type certificates, information verification and inventory are completed by the RFID card reader and OCR scanner combined recognition unit, which is fast and accurate. For book-type certificates, two inventory modes are provided for selection.

[0033] Fast inventory mode: Through the linkage of RFID and certificate storage compartment sensors, verify whether there is a certificate in each storage compartment and whether the certificate information is consistent with the system record. This inventory method is highly efficient, and 500 certificates can be inventoried within 10 minutes, generating inventory results of "normal / missing / abnormal", and quickly grasping the overall storage status of the certificates.

[0034] Precise inventory mode: Through the automatic transmission unit, certificates are retrieved one by one, and book-type certificates are first identified and classified by mode, and then the book-type certificate page turning unit is activated according to the certificate category. According to the established "certificate material-thickness-page turning parameter" three-dimensional mapping library, including friction wheel pressure, suction cup negative pressure value and page turning speed parameters for scenarios such as ultra-thin old passports (e.g., friction wheel pressure 0.2N, suction cup negative pressure 0.04MPa), sticker passports (e.g., friction wheel pressure 0.3N, suction cup negative pressure 0.05MPa), passports with protective cover (e.g., friction wheel pressure 0.35N, suction cup negative pressure 0.06MPa) and other scenarios. During page turning, the system automatically retrieves matching parameters to ensure accurate and error-free page turning. At the same time, the storage unit position is calibrated (e.g., adjust the storage compartment height according to the certificate thickness) during page turning, and the 2 million pixel industrial camera binocular vision positioning is carried out to analyze the certificate status. According to the thickness of the book-type certificate (e.g., passport 30-60 pages), the storage compartment spacing is automatically adjusted to ensure that the certificates are stored neatly and orderly. The certificate recognition unit reacquires the certificate information and marks the unqualified certificates (e.g., damaged, information changed) after comparing with the system archive data. At the same time, the visual camera synchronously detects the integrity of each page (e.g., whether the page is missing, whether the page is torn) and records the missing page number, generating a "single-page-level unqualified report". After page turning and detection are completed, the certificate is conveyed to the corresponding storage unit through the conveyor belt, and the system automatically generates an inventory report (e.g., including the number of qualified certificates, the number of unqualified certificates, and the number of missing certificates), and supports exporting Excel / PDF format, providing detailed and accurate data support for certificate management.

[0035] Figure 2 According to the structure of the certificate access cabinet shown in some embodiments of the present specification, Figure 2As shown, the top of the certificate access cabinet is provided with an indicator board for intuitively displaying the working status of the equipment, such as idle, running, fault and other information, so that the user can timely understand the equipment condition. The camera beside it can monitor the operation process in real time to ensure the safety and traceability of the access behavior. The angle adjustment module is arranged below the camera, which can flexibly adjust the angle of the camera according to the actual needs to ensure that the monitoring picture is clear and comprehensive. In the middle of the cabinet, a large-size touch screen provides a simple and intuitive operation interface for the user, and the user can easily complete the input of various instructions through touch operation. The barcode scanning area below the touch screen supports fast scanning of the barcode information on the certificate to realize fast identification and positioning of the certificate. The certificate outlet and the fingerprint instrument are arranged adjacent to each other. The certificate outlet is used to output the certificate taken by the user, and is reasonably designed to ensure smooth output of the certificate; the fingerprint instrument is used to collect and verify the fingerprint information of the user, and the biological recognition technology is used to enhance the accuracy and safety of identity verification. On one side of the certificate outlet, there is also a batch certificate storage entrance to facilitate the user to store multiple certificates at one time and improve the storage efficiency. The lower part of the cabinet is provided with a cabinet door with a lock as a channel for batch certificate taking. The cabinet door lock adopts a high safety design, and only authorized personnel can open it through a specific way to ensure the safety of the certificates in the cabinet. In addition, wheels are installed at the bottom of the cabinet to facilitate the movement and deployment of the equipment, and the position can be flexibly adjusted according to the actual needs.

[0036] The interaction module is configured to receive a certificate storage request or a certificate taking request initiated by a user, and the identity recognition module is configured to perform identity verification based on the certificate storage request or the certificate taking request initiated by the user.

[0037] The certificate access cabinet is configured to receive a certificate placed by a user in response to a certificate storage request initiated by the user after identity verification.

[0038] Specifically, when an ordinary user initiates a certificate storage request, the user needs to log in to the intelligent certificate management software system at the system end, select the certificate to be stored in the certificate management certificate list, generate a unique certificate storage two-dimensional code, and the self-service certificate storage needs to scan the two-dimensional code to go to the certificate access cabinet to store the certificate; if an administrator initiates a batch certificate storage request, the administrator can select batch associated certificate storage after logging in to the certificate management system, check the certificates to be stored in the certificate list, check the information, generate a batch certificate storage two-dimensional code, or select fast reading storage, read the certificate data through the certificate reader, and import the batch certificate storage two-dimensional code into the intelligent certificate management software system.

[0039] When an ordinary user initiates a certificate taking request, the user logs in to the intelligent certificate management software system at the system end, selects the certificate to be taken in the certificate management certificate list, selects the return time, clicks to confirm, and generates a certificate taking two-dimensional code after the authorized certificate holder's ID number.

[0040] For ordinary user storage and evidence collection, after going to the certificate management cabinet and entering the intelligent certificate management software system, whether it is storage or collection (for example, ID card, two-dimensional code, mobile phone message, etc.), it needs to follow the prompts to collect the portrait, start the binocular living body detection, and after the detection, it carries out the comparison between the person and the certificate. Only when the binocular living body detection and the comparison between the person and the certificate are passed, it is considered that the identity verification is passed. For the administrator storage, when going to the intelligent certificate management cabinet at the device end, it needs to scan the batch storage two-dimensional code generated at the software system end. The certificate storage and collection cabinet body automatically verifies the administrator's right and the information of the certificate to be stored. By comparing the number of the certificate to be stored with the current number of the self-adaptive storage grid of the certificate storage and collection cabinet body, when the self-adaptive storage grid is sufficient, it is considered that the identity verification is passed and the subsequent operation can be entered.

[0041] For ordinary user storage, after the identity verification is passed, the user selects the type of the certificate to be stored (for example, card type certificate, book type certificate), puts the certificate into the storage port according to the interface guidance, and carries out the storage operation. According to whether the user continues to store the certificate, it is determined whether the storage process is ended. After all the certificates are stored, the interface prompts “storage success” and synchronously updates the software system end certificate state to “in stock”.

[0042] For ordinary user collection, if the ordinary user does not collect the certificate within 24 hours after the collection two-dimensional code is generated, the software system automatically cancels the collection authorization, the certificate state is restored to “in stock”, and a prompt is pushed to the user and the administrator. The ordinary user needs to re-initiate the collection application, and the administrator can set the “overdue collection” number limit (maximum 3 times). If the number is exceeded, it needs to be authorized after manual approval.

[0043] For administrator storage, after the identity verification is passed and the batch storage interface is entered, the “certificate storage” of the cabinet machine interface is clicked, the batch storage port is automatically opened, the storage port is equipped with LED light belt to display different color prompts according to the type of the certificate, the interface displays the placement specification corresponding to the type of the certificate to be stored, the administrator places the certificate according to the prompt, and after the placement is completed, “confirm placement” is clicked.

[0044] The certificate analysis module is used for responding to the certificate storage request initiated by the user, and carrying out the abnormal detection and information identification of the certificate placed by the user.

[0045] Figure 3 It is the flowchart of the abnormal detection and information identification of the certificate placed by the user according to some embodiments of the present specification, as shown in Figure 3 The certificate analysis module is further used for: carrying out the inclination detection of the certificate placed by the user; after the inclination detection is passed, based on the certificate storage request initiated by the user, carrying out the quantity detection of the certificate placed by the user; After the quantity check passes, the document storage request initiated by the user is used to perform type matching check on the document placed by the user. After the type matching detection passes, the user-placed documents are checked by flipping through the pages outwards. After the page-turning outward detection passes, the stacking standard of the documents placed by the user is checked. After the stacking standard test is passed, the physical status of the documents placed by the user is checked based on the document storage request initiated by the user.

[0046] In some embodiments, the document analysis module is further used for: An infrared array sensor is used to generate a binarized image of the outline of the ID card placed by the user. Based on the contour binarized image of the document placed by the user, the four corner points of the document placed by the user are determined. The tilt detection of the user-placed document is performed based on the four corner points of the document.

[0047] Specifically, the infrared array sensor emits infrared light. When the light shines on the document, the sensor receives the reflected light signal and converts it into an electrical signal because the document and the surrounding environment have different reflective properties. After a series of processing steps, such as signal amplification, filtering, and threshold comparison, pixels with values ​​higher than the threshold are set to white (e.g., 255 represents the background area), and pixels with values ​​lower than the threshold are set to black (e.g., 0 represents the document outline area), thus obtaining a clear, binarized image.

[0048] like Figure 4 As shown, the image is scanned from left to right and from top to bottom to find the leftmost point A (x1, y1), and from right to left and from top to bottom to find the rightmost point B (x2, y2). Then, based on the relationship between the ordinates of points A and B, the lower left or lower right corner point is determined, and then the other two corner points are determined. These four corner points are the four vertices of the document outline, which define the position and shape of the document in the image.

[0049] Take vertically upwards The distance is used to obtain the boundary point. Calculate the tilt angle based on points A and C. :

[0050] in, It is an absolute value function.

[0051] The cabinet has a built-in photoelectric sensor that automatically counts the number of documents.

[0052] In some embodiments, the document analysis module is further used for: acquire an image of the certificate placed by the user; determine the type of the certificate placed by the user based on the image of the certificate placed by the user, wherein the type of the certificate placed by the user is a card type or a book type; perform type matching detection on the certificate placed by the user based on the certificate storage request initiated by the user and the type of the certificate placed by the user.

[0053] Specifically, based on the image of the certificate placed by the user, image recognition can be performed to determine the type of the certificate placed by the user, based on the certificate storage request initiated by the user, the type of the certificate to be placed by the user is determined, type matching detection is performed, and according to the type matching detection result, corresponding prompt information is issued.

[0054] For example: 1. Based on the image of the certificate placed by the user, it is determined that the type of the certificate placed by the user is a book type certificate, and the certificate storage request initiated by the user indicates that the type of the certificate to be placed by the user is to place a card type certificate. Then directly prompt "request is card type, actual is book type, please store card type certificate"; 2. Based on the image of the certificate placed by the user, it is determined that the type of the certificate placed by the user is a book type certificate, and the certificate storage request initiated by the user indicates that the type of the certificate to be placed by the user is to place a book type certificate. Then store correctly and perform next operation; 3. Based on the image of the certificate placed by the user, it is determined that the type of the certificate placed by the user is a card type certificate, and the certificate storage request initiated by the user indicates that the type of the certificate to be placed by the user is a card type certificate. Then store correctly and perform next operation; 4. Based on the image of the certificate placed by the user, it is determined that the type of the certificate placed by the user is a card type certificate, and the certificate storage request initiated by the user indicates that the type of the certificate to be placed by the user is to place a book type certificate. Then directly prompt "request is book type, actual is card type, please store book type certificate".

[0055] In some embodiments, the certificate analysis module is further used for: acquire an image of the certificate placed by the user; preprocess the image of the certificate placed by the user and perform segmentation to determine the area image of each certificate placed by the user; For each certificate placed by the user, based on the area image of the certificate, the straightness, inner page protrusion and thickness uniformity of the certificate are determined, and the page turning outward detection is performed.

[0056] Specifically, first, the image of the certificate placed by the user is acquired. Specifically, the side area of the stack of multiple passports is photographed to obtain an initial image containing the information of multiple passports, thereby providing basic data for subsequent analysis.

[0057] Next, the collected images are pre-processed. The first step is grayscale processing, which converts the RGB color image into a grayscale image, simplifies the image information, and reduces the amount of data for subsequent processing. Subsequently, the grayscale image is processed using Gaussian filtering (for example, using a 5x5 convolution kernel), which aims to eliminate light source noise. Since factors such as uneven light sources can introduce noise interference in the image, affecting the accuracy of subsequent analysis, Gaussian filtering can effectively smooth the image and reduce noise impact. After that, perform light correction, and perform contrast-limited adaptive histogram equalization on the filtered grayscale image. Through this operation, the contrast of the image can be improved, and the details in the image can be made clearer, especially the key information such as the edges of the passport can be better presented.

[0058] After preprocessing, image segmentation is performed to determine the area image of each certificate. First, perform morphological gradient and vertical gradient fusion operation to strengthen the irregular features of the page edge, making the edge features of the passport more prominent, which is convenient for subsequent segmentation. Then, an adaptive threshold is obtained to obtain an adaptive threshold image, and a native Otsu (Otsu's Method) segmentation is performed to obtain a global Otsu threshold segmentation image. Then, double-threshold fusion is performed, and the adaptive threshold image and the global threshold segmentation image are subjected to "and operation", only the region determined as foreground by both is retained, which can more accurately extract the edge information of the passport. Then, morphological operation is performed, first using a 5*5 kernel to do 2 times of closed operation iteration to fill the small gaps in the passport edge, making the edge more complete; then using a 3x3 kernel to do 1 time of open operation to eliminate small bright spots caused by dust, reflection and the like, further purifying the image. Finally, the maximum bounding rectangle is extracted to obtain the bounding rectangle of the certificate stacking area, excluding edge interference, and determining the approximate certificate stacking range.

[0059] In order to separate the independent side edge of each passport from the stacked side edge, single-passport segmentation is performed. First, use the Sobel operator to extract the vertical edge to highlight the vertical edge features of the passport; through threshold processing, some unimportant edge information is removed. Then, perform vertical projection to count the number of white pixels (edges) in each column (vertical direction) of the image, obtain the projection curve, and perform Gaussian smoothing on the projection curve to eliminate noise interference and find the local minimum points of the smoothed curve. These points correspond to the separation lines between books. Then, according to the prior knowledge of the certificate width (such as the minimum width threshold), filter out the nearby minimum points (which may be caused by dust, reflection and the like), and retain the effective segmentation boundary, so as to segment the side edge region of each passport.

[0060] For each passport after segmentation, the straightness, inner page bulge and thickness uniformity of the document are determined based on its region image, and then the page turning outward detection is performed. In terms of detecting straightness, the coordinates of the passport contour points are extracted, linear fitting is performed, the predicted value of the fitting straight line is calculated, and the vertical distance of each point to the fitting straight line is calculated. When the maximum distance is less than the set threshold, it means that the straightness of the passport is good. When detecting the inner page bulge, morphological dilation is first performed to highlight the bulge region, and then the height difference before and after dilation is calculated to determine whether the inner page has a bulge. When detecting thickness uniformity, vertical projection is performed, the sum of each column of pixels is calculated to reflect the width / thickness, and the effective column is found, for example, the column with a pixel sum greater than 0, which is the passport region. The maximum and minimum values of the thickness (width) are calculated, and the thickness change is the difference between the maximum and minimum values. When the straightness meets the standard, the bulge height is less than the set threshold, and the thickness change is less than the set threshold, it means that there is a binding edge, and the page turning outward detection fails. Through this series of operations, the document analysis module can comprehensively and accurately analyze the document placed by the user and determine whether the page is turned outward.

[0061] In some embodiments, the document analysis module is further used for: collecting an image of a document placed by a user; preprocessing the image of the document placed by the user to generate a preprocessed image; performing edge feature enhancement, double-threshold fusion and morphological post-processing on the preprocessed image to generate a twice-processed image; converting the twice-processed image to an HSV color space, counting the pixel ratio and stripe distribution characteristics, and performing page turning outward detection.

[0062] Specifically, the image of the document placed by the user is collected, specifically, the side edge region of a stack of multiple passports is photographed to obtain an initial image containing key information of the multiple passports. Then, the collected image is preprocessed to generate a preprocessed image. The first step is to perform grayscale processing to convert the RGB color image to a grayscale image, simplify the image information structure, and reduce the data complexity of subsequent processing. Subsequently, a 5x5 convolution kernel Gaussian filter is used to process the grayscale image, effectively eliminating noise caused by uneven light sources and other factors that interfere with image quality. Gaussian filtering can smooth the image and reduce the negative impact of noise on subsequent analysis. After that, light correction is performed, and the filtered grayscale image is subjected to contrast-limited adaptive histogram equalization to improve the image contrast and make the passport edge and other detailed information more clearly presented.

[0063] Then, the pre-processed image is subjected to edge feature enhancement, double threshold fusion and morphological post-processing to generate a second-processed image. First, morphological gradient and vertical gradient fusion operation is performed to enhance the irregular features of the page turning edge, making the passport edge features more prominent, facilitating subsequent segmentation and detection. Then, adaptive threshold is used to obtain an adaptive threshold image, and native Otsu segmentation is performed to obtain a global Otsu threshold segmentation image. Subsequently, double threshold fusion is performed, and the adaptive threshold image and the global Otsu threshold segmentation image are subjected to "and operation", only retaining the regions determined as foreground by both, to improve the accuracy of edge extraction. Subsequently, morphological operation is carried out, first using a 5*5 kernel to perform 2 iterations of closed operation to fill in the small gaps in the passport edge, making the edge more complete; then using a 3*3 kernel to perform 1 iteration of open operation to eliminate small bright spots caused by dust, reflection and the like, to purify the image. Finally, the maximum bounding rectangle is extracted to obtain the bounding rectangle of the certificate stack area, excluding edge interference and determining the approximate certificate stack range.

[0064] Finally, the second-processed image is converted to the HSV color space, and the pixel ratio and stripe distribution features are counted to detect the page turning outward. On the one hand, the dark pixel ratio is calculated, and the proportion of dark pixels in the foreground region is counted, wherein the dark pixels are the pixels with a value of 1 after binarization, and the proportion should be less than a threshold value when the page is turned outward. At the same time, morphological operations (such as erosion and dilation) are used to detect whether there is a large area of continuous dark block. When the proportion is less than the threshold value and there is no large dark block area, it indicates that the certificate placement may meet the requirements. On the other hand, stripe distribution feature verification is performed, the number of dark pixels in each column is counted, and the variance of the number of dark pixels in the column direction is calculated. The smaller the variance, the more uniform the stripes, which meets the characteristics of the page turning outward. Through the above comprehensive judgment, the detection of the certificate page turning outward is completed.

[0065] In some embodiments, the certificate analysis module is further used for: collecting an image of a certificate placed by a user and pre-processing the image to generate a pre-processed image; performing edge detection and contour extraction on the pre-processed image, and fitting a reference line and calculating a deviation for standardization detection.

[0066] Specifically, first, an image of the certificate placed by the user is collected and preprocessed to generate a preprocessed image. A clear certificate stacking area image is first obtained to ensure that the image accurately reflects the stacking state of the certificate. Next, the collected RGB image is converted to a grayscale image, which can reduce the subsequent calculation amount. Because a grayscale image only contains brightness information, it has a smaller data size than an RGB image, which can improve processing efficiency. Subsequently, the grayscale image is processed using a 5x5 convolution kernel Gaussian filter to effectively eliminate noise caused by uneven distribution of ambient light, which can interfere with image quality. Gaussian filtering can smooth the image, making it clearer and providing a good foundation for subsequent analysis.

[0067] Then, edge detection and contour extraction are performed on the preprocessed image, and a reference line fitting and deviation calculation are performed for stacking specification detection. First, Otsu adaptive thresholding is used for threshold segmentation to separate the certificate region from the background binary, making the certificate region and background form a clear contrast, which facilitates subsequent processing. Next, the Canny edge detection algorithm is used to extract the continuous contour of the top and left edge of the certificate. The Canny algorithm can accurately detect the edge information in the image, providing accurate data for contour extraction. Subsequently, contour screening is performed. Small noise contours are excluded based on area to avoid interference from irrelevant small regions. Target contours that match the certificate size range are retained based on the aspect ratio to ensure that the extracted contours are the true contours of the certificate.

[0068] After contour extraction, reference line fitting and deviation calculation are performed. The least squares method is used to fit a straight line to the left edge contour, and the straight line is used as the reference line for stacking, providing a reference standard for subsequent deviation calculation. The vertical distance from all points on the contour to the reference line is traversed to find the maximum value, which is the vertical stacking deviation. Finally, the calculated vertical stacking deviation is compared with a preset threshold value. If the deviation is within the preset threshold value (for example, the deviation of the present certificate is ≤2mm, and the card certificate has no overlap and no tilt), the qualified result is output; if it exceeds the preset threshold value, the abnormal result is output. When it does not meet the specification, the transmission channel is paused, the cabinet interface displays "placement error: XX type certificate orientation / stacking error", and specific adjustment suggestions are marked, and the certificate storage port is exited.

[0069] In some embodiments, the certificate analysis module is further used for: collecting an image of the certificate placed by the user, and detecting damage, stains, and foreign matter; detecting the chip of the certificate placed by the user; based on the certificate storage request initiated by the user, performing information matching detection on the certificate placed by the user.

[0070] Specifically, the image of the certificate placed by the user is collected, damage, stain and foreign matter detection is performed, and whether the certificate is damaged is automatically identified by using a ResNet50 model, such as edge tearing, corner missing, etc.; whether there is a stain to block is accurately judged, especially the stain covering the key information such as the certificate number and the valid period is investigated in key; at the same time, whether there is foreign matter carried, such as a paperclip, a sticker, etc. can be effectively detected, so as to ensure that the certificate surface is complete and the information is clear and unobstructed.

[0071] Secondly, the chip of the certificate placed by the user is detected, and this detection is mainly aimed at the card and the card two-in-one certificate. The NFC card reader is used to automatically read the chip information of the certificate. On the one hand, it detects whether the chip is readable, and if the chip cannot be normally read, it is determined that the chip is abnormal; on the other hand, the certificate number in the chip is compared with the number pre-stored in the system end, and if the two are inconsistent, it is also determined that the chip is abnormal, so as to ensure the accuracy and effectiveness of the chip information.

[0072] Further, based on the certificate storage request initiated by the user, the information matching detection of the certificate placed by the user is performed. After the physical detection is passed, the validity period information read by the chip or the reader is used to judge whether it is within the validity period, and if the certificate has expired, it is determined that the "time limit is unqualified"; then the information matching detection is performed, the name and certificate number of the certificate holder printed on the surface of the certificate are compared with the information of the certificate holder associated with the system end, and if the two are inconsistent, it is determined that the "information is not matched", which may be caused by the user's mistake.

[0073] For the certificate unqualified in each level of detection, the automatic marking function is used to mark the unqualified type, and the independent sorting channel is used to transport it to the abnormal certificate recycling port. At the same time, the unqualified certificate list is displayed on the interface of the cabinet machine, including the certificate number, the unqualified type and the processing suggestion, so that the user can understand the certificate problem in time and take corresponding measures.

[0074] The certificate storage cabinet is used for the result of the abnormal detection and information recognition based on the certificate analysis module.

[0075] The certificate storage cabinet is also used to take out the stored certificate in response to the certificate taking-out request initiated by the user after the identity verification is passed.

[0076] The intelligent certificate management system of multiple types of entry and exit certificates based on image processing can also include other functional modules.

[0077] In terms of information analysis, the system has strong data statistics and use behavior analysis capabilities. The data statistics function can automatically count the total number of certificates, the number of loans, and the number of in-stock quantities, and can also calculate the proportion of each type of certificate (such as passports and Hong Kong and Macao travel permits), and generate daily, weekly, and monthly reports, supporting data filtering by department and personnel dimensions to help managers fully understand certificate usage. Use behavior analysis focuses on certificate loan frequency, average loan duration, and high-frequency loan personnel, and can accurately identify abnormal use, such as frequent loaning of a certificate in a short period of time, generating an analysis report and marking the risk item, providing strong evidence for security management.

[0078] The intelligent certificate management system for multiple types of entry and exit certificates based on image processing can also include a security warning module to ensure the safe operation of the system. The expiration reminder function reminds the applicant through system pop-up windows and mobile phone messages 15 days, 7 days, and 3 days before the expected return time set by the loan; if the certificate is not returned on time, the reminder is upgraded to the administrator and includes the number of overdue days and certificate information. Abnormal warning covers certificate abnormalities and operation abnormalities. When a certificate is detected to be unqualified (damaged, expired), or missing during inventory, an early warning is sent to the administrator, including the abnormal certificate number and type. In terms of operation abnormalities, unauthorized attempts to access trigger general warnings, transmission channel jams trigger more serious warnings, and illegal opening of the cabinet triggers high-risk warnings, corresponding to different volume sound and light alarms. High-risk warnings also synchronize with the unit's security system and push to the security terminal.

[0079] The intelligent certificate management system for multiple types of entry and exit certificates based on image processing can also include a loan registration module to realize electronic operation. The loan registration module displays the loan application form through the touch screen, supports filling in the loan reason and expected return time, and automatically associates the applicant and certificate information after submission to generate an electronic registration record, eliminating manual paper registration. The approval linkage component supports multi-level approval processes for classified units. After the applicant submits the application, the system automatically pushes the approval request to the mobile terminal of the approver, and the certificate can only be loaned after the approval is passed.

[0080] The intelligent certificate management system for multiple types of entry and exit certificates based on image processing can also include a state query module to enable administrators to monitor certificate status in real time. The real-time query function enables administrators to query the in-stock location, loan status, loan personnel, and loan duration within 1 second through the touch screen or the back-end system. The positioning navigation component provides positioning and navigation services for in-stock certificates. The administrator inputs the target certificate number, and the system displays the storage location and lights up the corresponding indicator light, making it easy and fast to retrieve the certificate.

[0081] The intelligent certificate management system for multiple types of entry and exit certificates based on image processing can also include a handover record module, which standardizes the certificate handover process of administrators and ordinary users. When handing over, the information of both parties is entered through the touch screen, the system automatically generates an electronic handover sheet containing certificate information, handover person, time, and remarks, supports electronic signature confirmation, and synchronously encrypts the handover record and operation record for traceability and export.

[0082] The intelligent certificate management system for multiple types of entry and exit certificates based on image processing can also include a data interaction module to ensure smooth data flow within and outside the system. The internal interaction unit is connected to the internal personnel information system of the unit through an API interface, synchronizes personnel information, and ensures accurate association between certificates and personnel. The external docking unit supports real-time linkage with the entry and exit management system, synchronizes data such as certificate validity period, automatically caches data when the network is interrupted, synchronizes preferentially after recovery, and pushes warning to the administrator in case of synchronization failure. The remote management unit allows the administrator to remotely view the cabinet status and other information through the background system (Web and mobile), and remotely authorizes special personnel to access the certificate.

[0083] The intelligent certificate management method for multiple types of entry and exit certificates based on image processing can include the following steps: In response to a certificate storage request initiated by a user, identity verification is performed, and after the identity verification is passed, in response to the certificate storage request initiated by the user, the certificate placed by the user is received, the certificate placed by the user is detected for abnormalities and information is recognized, and based on the results of the abnormality detection and information recognition, the certificate is stored; In response to a certificate retrieval request initiated by a user, the stored certificate is retrieved.

[0084] For more description of the intelligent certificate management method for multiple types of entry and exit certificates based on image processing, refer to the related description of the intelligent certificate management system for multiple types of entry and exit certificates based on image processing, which will not be repeated here.

[0085] Finally, it should be understood that the embodiments described in the specification are only used to illustrate the principles of the embodiments of the specification. Other variations can also be within the scope of the specification. Therefore, as an example but not limitation, alternative configurations of the embodiments of the specification can be considered consistent with the teachings of the specification. Accordingly, the embodiments of the specification are not limited to the embodiments explicitly introduced and described in the specification.

Claims

1. An intelligent document management system for multiple types of entry and exit documents based on image processing, characterized in that, This includes an interactive module, a document storage cabinet, and an identity recognition module and a document analysis module set up inside the document storage cabinet; The interaction module is used to receive a document storage request or document retrieval request initiated by the user, and the identity recognition module is used to perform identity verification based on the document storage request or document retrieval request initiated by the user. The document storage cabinet is used to receive the documents placed by the user in response to the user's document storage request after the identity verification is passed. The document analysis module is used to respond to the document storage request initiated by the user and to perform anomaly detection and information recognition on the documents placed by the user. The document storage cabinet is used to store documents based on the results of anomaly detection and information recognition from the document analysis module. The document storage cabinet is also used to retrieve the stored documents in response to a user's document retrieval request after identity verification.

2. The intelligent document management system for multiple types of entry and exit documents based on image processing according to claim 1, characterized in that, The document analysis module is further used for: Perform tilt detection on the documents placed by the user; After the tilt detection passes, the number of documents placed by the user is checked based on the document storage request initiated by the user. After the quantity check passes, the document storage request initiated by the user is used to perform type matching check on the document placed by the user. After the type matching detection passes, the user-placed documents are checked by flipping through the pages outwards. After the page-turning outward detection passes, the stacking standard of the documents placed by the user is checked. After the stacking standard test is passed, the physical status of the documents placed by the user is checked based on the document storage request initiated by the user.

3. The intelligent document management system for multiple types of entry and exit documents based on image processing according to claim 2, characterized in that, The document analysis module is further used for: An infrared array sensor is used to generate a binarized image of the outline of the ID card placed by the user. Based on the contour binarized image of the document placed by the user, the four corner points of the document placed by the user are determined. The tilt detection of the user-placed document is performed based on the four corner points of the document.

4. The intelligent document management system for multiple types of entry and exit documents based on image processing according to claim 2, characterized in that, The document analysis module is further used for: Capture images of the documents placed by the user; Based on the image of the document placed by the user, determine the type of document placed by the user, wherein the type of document placed by the user is card or booklet; Based on the user's request to store the document and the type of document placed by the user, a type matching detection is performed on the document placed by the user.

5. The intelligent document management system for multiple types of entry and exit documents based on image processing according to claim 2, characterized in that, The document analysis module is further used for: Capture images of the documents placed by the user; The images of the documents placed by the user are preprocessed and segmented to determine the area image of each document placed by the user. For each document placed by the user, based on the regional image of the document, the straightness, inner page protrusion, and thickness uniformity of the document are determined, and page turning outward detection is performed.

6. The intelligent document management system for multiple types of entry and exit documents based on image processing according to claim 2, characterized in that, The document analysis module is further used for: Capture images of the documents placed by the user; The image of the ID card placed by the user is preprocessed to generate a preprocessed image; The preprocessed image is subjected to edge feature enhancement, dual threshold fusion, and morphological post-processing to generate a secondary processed image; The image after secondary processing is converted to the HSV color space, and the pixel ratio and stripe distribution characteristics are statistically analyzed to perform page-turning outward detection.

7. The intelligent document management system for multiple types of entry and exit documents based on image processing according to any one of claims 2-6, characterized in that, The document analysis module is further used for: Collect images of the user's placed identification documents and preprocess them to generate preprocessed images; Edge detection and contour extraction are performed on the preprocessed image, and baseline fitting and deviation calculation are performed for superimposed standard detection.

8. The intelligent document management system for multiple types of entry and exit documents based on image processing according to any one of claims 2-6, characterized in that, The document analysis module is further used for: Collect images of the user's placed identification documents and detect damage, stains, and foreign objects; Perform chip detection on the user's placed identification document; Based on the user's request to store documents, information matching and detection are performed on the documents placed by the user.

9. The intelligent document management system for multiple types of entry and exit documents based on image processing according to any one of claims 1-6, characterized in that, The document storage cabinet includes a document storage unit, an adaptive storage unit, an automatic transmission unit, a booklet document flipping unit, and an automatic inventory unit. The adaptive storage unit includes multiple adaptive document storage compartments with adjustable sizes. The automatic transmission unit is used to transfer documents from the document storage unit to the adaptive storage unit or vice versa. The booklet document flipping unit is used to flip pages of booklet documents. The automatic inventory unit is used to automatically count the stored documents.

10. A smart document management method for multiple types of entry and exit documents based on image processing, characterized in that, The intelligent document management system for multiple types of entry and exit documents based on image processing, as described in claim 1, comprises: In response to a user's request to store documents, the system performs identity verification. After successful identity verification, it receives the documents placed by the user, performs anomaly detection and information recognition on the documents placed by the user, and stores the documents based on the results of anomaly detection and information recognition. In response to a user's request to retrieve documents, the stored documents are retrieved.

Images