An entry and exit certificate verification method based on dynamic weight and multi-modal fusion

Abstract

The application relates to an entry and exit certificate verification method based on dynamic weight and multi-modal fusion, and belongs to the technical field of entry and exit certificate information processing and verification. The method comprises the following steps: collecting multi-source heterogeneous data of a certificate through visual reading, machine reading and chip reading; constructing a verification data graph with a unique certificate identifier as the core to associate and align the data; based on a dynamic weight distribution model, performing weighted voting fusion on homologous key fields and performing semantic fusion on heterogeneous complementary fields; constructing a traceable verification decision factor matrix, determining the verification conclusion according to the matrix content, positioning the root cause in an abnormal case, and generating a structured verification record and a disposal scheme containing a complete decision path. Through dynamic weight fusion and a decision factor matrix, the application effectively solves the problem of fragmentation of the existing verification method, and significantly improves the accuracy, reliability and process traceability of the verification result.

Description

Technical Field

[0001] This invention belongs to the field of entry and exit document information processing and verification technology, specifically involving an entry and exit document verification method based on dynamic weighting and multimodal fusion. Background Technology

[0002] With increasingly frequent cross-border flows of people globally, border inspection verification of entry and exit documents is a core component of security control and customs clearance, urgently requiring accuracy and efficiency. Currently, border inspection mainly relies on three verification methods: visual reading, machine reading, and chip reading. These three methods constitute a basic system but are fragmented and cannot coordinate efficiently.

[0003] Visual verification relies on manual inspection of document surface information and consistency between the person and the document. While it can identify physical anti-counterfeiting features, it is inefficient, prone to misjudgments due to human experience or fatigue, and the fragmented information is difficult to correlate. Machine reading improves efficiency and automation by parsing the machine-readable code on the document, but it has limited information capacity, is susceptible to environmental interference leading to parsing distortion, and lacks a data quality assessment mechanism. Chip reading can extract encrypted biometric features and depth data, offering the highest security, but it relies on dedicated equipment, and a reading failure interrupts the process. Furthermore, the data is isolated from the previous two methods, making it difficult to form a unified verification basis.

[0004] The drawbacks of operating these three methods in a fragmented manner are prominent: data is stored in a scattered manner without an integration mechanism, verification relies on a single data source, and the complementarity is poor; when multiple data sources conflict, it is impossible to quantify the reliability of each source and it is difficult to locate the root cause of the anomaly.

[0005] With the surge in cross-border flows, traditional methods struggle to balance security and efficiency. Meanwhile, advancements in document forgery technology pose challenges to single-method verification. Therefore, breaking down technological barriers, quantifying data reliability through dynamic weighting, and integrating information via multimodal fusion to generate complete and traceable verification records have become core issues urgently needing resolution in border inspection. Summary of the Invention

[0006] To address the aforementioned problems in the existing technology, this invention provides a method for verifying entry and exit documents based on dynamic weights and multimodal fusion. The objective of this invention can be achieved through the following technical solution: A method for verifying entry and exit documents based on dynamic weights and multimodal fusion includes: S1: Obtain visual information from the surface of the document, extract document text, facial regions, and physical anti-counterfeiting features; read the machine-readable code of the document, parse standardized data fields, and score the data quality of the code reading by computer; extract digital biometric features from the chip and verify the validity of the chip's digital signature; S2: Construct a verification data graph with the unique identifier of the document as the root node, and mount visually readable, machine-readable, and chip data as attribute nodes; use entity parsing algorithms to perform fuzzy matching based on the core fields of name, date of birth, and nationality, and output multi-source associated aligned data; S3: Based on the multi-source associated alignment data, construct a dynamic weight allocation model, perform weighted voting fusion on key fields from the same source, and perform semantic fusion on complementary fields from different sources to generate an issuance record that integrates coding and natural language information; S4: Based on the results of the weighted voting fusion and semantic fusion, a verification decision factor matrix is ​​constructed. The verification decision factor matrix records the final values ​​of key fields, the data sources on which they are based, and their dynamic weights. Based on the consistency status and weights of the fields in the verification decision factor matrix, the verification conclusion is determined. If the verification conclusion is abnormal, the root source data source of the abnormality is located. Based on the location result and the decision path recorded in the verification decision factor matrix, a structured border inspection entry and exit verification record and the corresponding handling plan are generated in combination with the issuance record.

[0007] As a preferred technical solution of the present invention, the method for verifying the validity of the digital signature of the chip is as follows: a certificate chain is constructed using a pre-set root certificate of the issuing authority, and decryption and hash value comparison operations are performed on the digital signature read from the chip.

[0008] Specifically, the verification data graph includes: the verification data graph is a directed attribute graph, wherein the unique identifier of the document serves as the central entity node, and visually readable, machine-readable, and chip data serve as feature nodes connected to it through inclusion relationships. Each feature node stores the original data and the extracted feature vector in the form of key-value pairs.

[0009] Specifically, the entity parsing algorithm is used to perform fuzzy matching based on the core fields of name, date of birth, and nationality. This includes: using a composite comparison algorithm based on text similarity and speech similarity, where the text similarity is calculated based on character edit distance and the speech similarity is calculated based on the pinyin or phonetic mapping relationship of double-byte characters; and setting a dynamically adjustable threshold range, which is adjusted based on the current data quality score of the corresponding data source.

[0010] Specifically, the dynamic weight allocation model includes: the dynamic weight coefficient is calculated from the inherent basic weight of the data source and the data quality score weight of the data collected this time; wherein, the preset relationship of the inherent basic weight is: the basic weight of chip-read data > the basic weight of machine-readable data > the basic weight of visually read data.

[0011] Specifically, the weighted voting fusion of key fields from the same source includes: for the same key field, aggregating the values ​​provided by all data sources and their respective dynamic weights, and adopting the value with the highest weight as the fusion result value; if there is a tie for the highest weight, the value provided by the data source with the higher inherent basic weight is adopted first.

[0012] Specifically, the semantic fusion of heterogeneous complementary fields includes: using named entity recognition and date normalization technology to parse the text description in the chip-issued information, mapping and associating the organization name, location, and date entities with the corresponding standard codes in the machine-readable code, and generating an enhanced issuance record that simultaneously contains structured encoding and readable text.

[0013] Specifically, the verification decision factor matrix includes: the verification decision factor matrix is ​​a two-dimensional relational table with key fields as rows and decision attributes as columns, and the decision attributes include: the final fusion value of the field, the original proposal value of each data source, the unique identifier of the adopted data source, the dynamic weight calculation result of the adopted data source in this verification, and the field consistency status marker.

[0014] Specifically, the method for locating the source of the anomaly is as follows: in the verification decision factor matrix, scan the rows of fields marked as "conflict" or "missing" in terms of consistency status, extract all corresponding source data identifiers and weights, and determine the source identifier with the lowest dynamic weight that provides an error or invalid value as the source of the anomaly.

[0015] Specifically, the generation of the handling plan includes: forming a query key by combining the type of the abnormal root source data source with the category of the abnormal field, and matching it in a predefined handling strategy rule base; the handling strategy rule base contains a standardized handling process instruction set for different scenarios.

[0016] The beneficial effects of this invention are as follows: Strengthening the accuracy and security of data collection lays a solid foundation for verification. Reliability is quantified and analyzed through machine-readable data quality scoring, and data quality is visualized and assessed by combining edge deformation rate and bit error rate, avoiding interference from invalid data. Simultaneously, a certificate chain is used to verify the chip's digital signature, ensuring the legality and integrity of the chip data from the source. This solves the problem of prioritizing acquisition over verification in existing technologies, providing a highly reliable data source for subsequent verification.

[0017] This approach enhances the efficiency and adaptability of multi-source data association, breaking down technological barriers. The constructed verification data graph uses the unique identifier of the document as its core, enabling the structured mounting of visually readable, machine-readable, and chip data. The entity parsing algorithm integrates text and speech similarity comparisons, along with dynamic threshold adjustments, which not only improves the accuracy of fuzzy matching but also balances automation and rigor through manual verification marking, effectively solving the pain points of fragmented multi-source data that are "difficult to align and prone to mismatch."

[0018] To ensure the reliability and self-optimization capabilities of data fusion and improve verification quality, a dynamic weighting model integrates the basic attributes and real-time quality of data sources, making the weighted voting results of fields from the same source more closely reflect real-world scenarios—high weighting of chip data ensures the accuracy of core information, while machine-readable and visually-readable data complement each other to cover details; semantic fusion of fields from different sources generates "structured + readable" issuance records through entity recognition and encoding mapping, balancing the needs of system interaction and manual review. Simultaneously, the re-collection of conflicting data sources enables dynamic optimization of the verification system.

[0019] This system ensures traceability of verification decisions and standardization of anomaly handling, reducing management risks. The verification decision factor matrix fully records the basis for field values, weights, and consistency status, making the entire decision-making process traceable. Anomaly root cause localization uses a dual screening process of weights and data validity to accurately pinpoint the problematic data source, avoiding blind investigation. Furthermore, the standardized handling scheme based on a rule base standardizes the anomaly handling process, solving the problems of existing technologies that rely on human experience and have inconsistent results. Attached Figure Description

[0020] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0021] Figure 1 This is a flowchart illustrating an entry / exit document verification method based on dynamic weights and multimodal fusion according to the present invention. Figure 2 This is a diagram of the dynamic weight fusion architecture of the present invention; Figure 3 This is a diagram of the verification decision factor matrix architecture of the present invention. Detailed Implementation

[0022] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0023] Please see Figure 1-3A method for verifying entry and exit documents based on dynamic weights and multimodal fusion, wherein the multimodal fusion includes the fusion of text modality (document text, machine-readable code data), image modality (visual information of the document, on-site image of the document holder), and biometric modality (chip-based digital biometric features), including: S1: Obtain visual information from the surface of the document, extract the document text, face region, and physical anti-counterfeiting features; read the machine-readable code of the document, parse standardized data fields, and score the data quality of the machine-readable code by computer. Specifically, optical distortion correction parameters are calculated by analyzing the edge distortion rate of the machine-readable code image, and the signal-to-noise ratio is calculated by comparing the bit error rate of the parsed data. The two are quantified into a data quality score for the machine-readable data; extract the digital biometric features within the chip to verify the validity of the chip's digital signature. S2: Construct a verification data graph with the unique identifier of the document as the root node, and mount visually readable, machine-readable, and chip data as attribute nodes; use entity parsing algorithms to perform fuzzy matching based on the core fields of name, date of birth, and nationality, and output multi-source associated aligned data; S3: Based on the multi-source associated alignment data, construct a dynamic weight allocation model, perform weighted voting fusion on key fields from the same source, and perform semantic fusion on complementary fields from different sources to generate an issuance record that integrates coding and natural language information; S4: Based on the results of the weighted voting fusion and semantic fusion, a verification decision factor matrix is ​​constructed. The verification decision factor matrix records the final values ​​of key fields, the data sources on which they are based, and their dynamic weights. Based on the consistency status and weights of the fields in the verification decision factor matrix, a verification conclusion is determined. If the verification conclusion is abnormal, the root source of the abnormality is located through the data source identifier and weight information recorded in the matrix. Based on the location result and the decision path recorded in the verification decision factor matrix, a structured border inspection entry and exit verification record and a corresponding handling plan are generated in conjunction with the issuance record.

[0024] This embodiment uses the ordinary passport entry verification at an international airport border checkpoint as a scenario. The verification system integrates visual reading, machine reading (ICAO 9303 standard), and chip reading (ISO 14443 protocol). The implementation steps are as follows: The visual reading module collects passport text (name "Zhang San", passport number "EFGHIJKL", etc.), facial region, anti-counterfeiting features, and on-site image of the certificate holder. After manual verification, it outputs visual reading data. The machine reading module parses the MRZ code to obtain standardized fields and calculates the data quality score through deformation rate and bit error rate. The chip module extracts digital biometric features (H-dimensional facial vector, J fingerprint feature points) and issuance information to verify the validity of the chip signature.

[0025] A verification data graph is constructed with the passport number "EFGHIJKL" as the core, and three types of data are attached. The core fields such as "name and date of birth" are compared by a text + voice composite algorithm. Dynamic thresholds are set according to data quality, and multi-source associated aligned data is output (the labeled fields match "consistent").

[0026] A dynamic weighting model (base weight + quality score) is constructed to calculate the chip data weights as follows: X1% for machine-readable data, X2% for machine-readable data, and X3% for visual data, where X1 > X2 > X3, and X1 + X2 + X3 = 100. However, the amount of information read is greater for visual data than for chip data, which is greater than for machine-readable data. Fields from the same source (such as name) are fused according to their weights, while fields from different sources (such as issuing authority) are semantically fused to generate an enhanced record containing the code "M".

[0027] A decision factor matrix is ​​constructed to record core field information. The system determines "verification passed" and generates a structured record containing information such as verification time and port number by combining the matrix with the issuance record. The handling plan is "direct release".

[0028] Among them, digital biometrics refers to the biometric information of the certificate holder extracted from the certificate chip and stored in the form of digital signals. In this embodiment, it is an H-dimensional facial feature vector (quantitatively describing the spatial relationship of key facial feature points) and fingerprint data containing J feature points. Compared with traditional photos, it is easier for computers to accurately compare and has the property of being unforgeable.

[0029] Multi-source associated alignment data is an integrated data set formed by matching and associating semantically related fields from three types of heterogeneous data—visual, machine-readable, and chip—with the unique identifier of the document as the core. In this embodiment, "Name-Zhang San" has the same semantics in the three types of data and is associated as the same field after algorithm verification. The chip's unique "Issuing Authority" is also included after being associated with the system's pre-stored code, thus solving the problem of "dispersed and disordered" multi-source data.

[0030] The final value is the determined result after the fusion of multi-source data. For example, the final value of "validity period of certificate" is "20XX year XX month XX day" which is consistent with the machine-readable and chip data. The data source on which it is based is the valid data source participating in the fusion of this field. For example, the name field covers three types of data sources. The dynamic weight is a reliability coefficient calculated by combining the inherent attributes of the data source and the quality score. For example, chip data receives the highest weight of X1% due to its high security, ensuring the accuracy of the fusion result.

[0031] Specifically, the method for verifying the validity of the digital signature on the chip is as follows: a certificate chain is constructed using a pre-set root certificate of the issuing authority, and decryption and hash value comparison operations are performed on the digital signature read from the chip.

[0032] In this embodiment, the process of building a certificate chain using a pre-set root certificate from an issuing authority is as follows: During the deployment phase, the border inspection verification system obtains an official root certificate (usually in X.509 standard format) from authoritative document issuing authorities such as the National Immigration Administration. This root certificate is signed with the issuing authority's exclusive private key and is the "source" of the trust system, containing core fields such as the issuing authority's identifier, public key information, and validity period. The system stores the root certificate in a dedicated security chip or encrypted database, granting only read access to prevent tampering or theft. During system initialization, the pre-set root certificate is set as the "trust anchor," meaning that the root certificate itself is completely trustworthy by default, and the legitimacy verification of all subsequent certificates is traced back to this root certificate, laying the foundation of trust for the certificate chain.

[0033] The border inspection system uses a chip reading module to extract the "terminal certificate" bound to the digital signature from the chip embedded in the entry / exit document. This certificate is generated exclusively for the current document by the issuing authority and contains the document chip identifier, the corresponding public key, and issuance information. It is the certificate directly used to decrypt the chip's digital signature. If the issuing authority adopts a multi-level issuing architecture of "root certificate-intermediate certificate-terminal certificate" (most authoritative institutions use this mode to improve security), the system will first read the pre-installed intermediate certificate from the chip's storage area; if the chip does not store it, the system will obtain the corresponding intermediate certificate from the issuing authority's certificate publishing server through a secure network (the intermediate certificate is signed by the root certificate and used to sign the terminal certificate).

[0034] The certificate chain is constructed following a bottom-up hierarchical verification logic, forming a complete trust chain of "terminal certificate → intermediate certificate (optional) → root certificate", specifically: Terminal certificate and intermediate certificate association verification: The system extracts the "issuer information" from the terminal certificate and matches it with the "subject information" of the intermediate certificate to confirm that the terminal certificate was issued by the authority corresponding to the intermediate certificate. Then, using the public key in the intermediate certificate, the system decrypts the digital signature of the terminal certificate and compares the hash value obtained after decryption with the hash value calculated from the terminal certificate's own information. If they match, the terminal certificate is trustworthy. Intermediate certificate and root certificate association verification (on demand): If an intermediate certificate exists, the system repeats the above logic—matching the "issuer information" of the intermediate certificate with the "subject information" of the root certificate, using the public key of the pre-set root certificate to decrypt the digital signature of the intermediate certificate and verify the hash value to confirm the trustworthiness of the intermediate certificate.

[0035] The certificate chain is considered valid when the terminal certificate (via an intermediate certificate) is successfully traced back to the pre-configured root certificate and each level of signature verification passes. The system synchronously checks the validity period (not expired) and revocation status (confirmed not to be revoked through the issuing authority blacklist) of all certificates in the certificate chain to ensure that the certificate chain is actually valid at the current point in time.

[0036] The certificate chain constructed through the above process ensures that the digital signature read from the chip by the border inspection system originates from an authoritative issuing authority, providing a legitimate public key basis for subsequent signature decryption and hash value comparison, and ultimately verifying the authenticity and integrity of the chip data.

[0037] Specifically, the verification data graph includes: the verification data graph is a directed attribute graph, wherein the unique identifier of the document serves as the central entity node, and visually readable, machine-readable, and chip data serve as feature nodes connected to it through inclusion relationships. Each feature node stores the original data and the extracted feature vector in the form of key-value pairs.

[0038] Specifically, the entity parsing algorithm, based on the core fields of name, date of birth, and nationality, performs fuzzy matching. This includes: employing a composite comparison algorithm based on text similarity and voice similarity, where text similarity is calculated based on character edit distance, and voice similarity is calculated based on the pinyin or phonetic mapping relationship of double-byte characters; and setting a dynamically adjustable threshold range, which is adjusted based on the current data quality score of the corresponding data source. In the fuzzy matching stage of border inspection document verification, the "dynamically adjusted threshold range" is an optimization design compared to the traditional "fixed threshold." Its core is to allow the field similarity judgment standard (i.e., the "threshold range") to dynamically change with the reliability of the data source, rather than remaining static. Essentially, it assesses the credibility of the data source through the "current state" dimension, balancing verification efficiency and accuracy.

[0039] In this embodiment, the threshold range is a "fuzzy zone" for determining the field matching result, rather than a single numerical value. For example, if the similarity of the name field is ≥A (upper threshold), it is directly determined as "matching correctly"; if it is ≤B (lower threshold), it is directly determined as "matching conflict". "Dynamic adjustment" means that the values ​​of A and B will change in real time according to the characteristics of the data source (visual / machine-readable / chip) currently participating in the matching, rather than using the same set of A and B values ​​in all scenarios.

[0040] The core logic of adjusting the threshold range is: the more reliable the data source, the looser the threshold range (reducing unnecessary human intervention); the less reliable the data source, the narrower the threshold range (improving the accuracy of manual review). The "data quality score" is the core indicator for quantifying the "reliability of the data source".

[0041] The data quality score refers to the data integrity, clarity, and other quality indicators of the data source itself in this verification, and is a reflection of the "current reliability" of the data source (corresponding to the machine-readable data quality score and visual anti-counterfeiting feature verification results in S1 above).

[0042] Different data sources have their own scoring dimensions—machine-readable data references optical distortion correction parameters and signal-to-noise ratio (e.g., F-score, G-score out of 5); visually readable data references document image clarity and the completeness of anti-counterfeiting feature extraction; chip data references signature verification results and biometric feature extraction success rate. All scores are uniformly normalized to a standard score of 0-100. The higher the quality score, the more reliable the current data, and the wider the threshold range can be; the lower the score, the narrower the range needs to be to avoid making incorrect judgments using low-quality data.

[0043] The system completes the adjustment in three steps: "basic interval → weight adjustment → final interval", ensuring that the logic is traceable.

[0044] Specifically, the dynamic weight allocation model includes: the dynamic weight coefficient W is calculated by the inherent basic weight W_b of the data source and the data quality score weight W_q of the data collected this time, according to the preset formula W=α*W_b+β*W_q, where α and β are configurable harmonic coefficients; wherein, the preset relationship of the inherent basic weights is: basic weight of chip read data > basic weight of machine read data > basic weight of visual read data.

[0045] Specifically, the weighted voting fusion of key fields from the same source includes: for the same key field, aggregating the values ​​provided by all data sources and their respective dynamic weights, and adopting the value with the highest weight as the fusion result value; if there is a tie for the highest weight, the value provided by the data source with the higher inherent basic weight is adopted first.

[0046] This embodiment uses the verification of electronic ordinary passports at a land border checkpoint as a scenario, focusing on the handling of fusion conflicts in the key field of "date of birth". The verification system is preset with a reliability threshold of Y% (Y is a preset value, representing the weight threshold for the data source to be basically reliable), an upper limit of N times for re-collection (N=A, A is a preset positive integer), and an initial basic weight relationship of data source as chip data > machine-readable data > visually readable data.

[0047] After completing the multi-source data collection of the passport holder, the system outputs the "Date of Birth" field values ​​and dynamic weights for each data source. When the system merges the "Date of Birth" field from the same source, it discovers a conflict between the visually readable data field value and the chip / machine-readable data (inconsistent months), triggering the conflict data source handling mechanism.

[0048] The system extracts the dynamic weight X3% of the conflicting data source (visual data) and compares it with the preset reliability threshold Y%. If X3% < Y% (the reliability of the visual data is insufficient this time), it meets the condition of "triggering the re-acquisition process". The system starts the "conflicting data source targeted acquisition" mode—controlling only the visual reading module to focus on the passport birth date printing area, automatically adjusting the camera focus and fill light intensity to capture a high-definition local image (avoiding the problem of insufficient clarity in the first acquisition), and simultaneously prompting the operator to assist in confirming the field position. After the first re-acquisition, the visual reading module clearly identifies the field value through the image enhancement algorithm, which is consistent with the chip and machine-readable data, the conflict is resolved, and the re-acquisition process terminates (the actual number of acquisitions is 1, which does not exceed the upper limit N=A).

[0049] The system generates a conflict handling log in the background, recording core information: conflict field (date of birth), conflict data source (viewing module), conflict reason (insufficient clarity of the first image acquisition led to recognition error), number of re-acquisitions (1 time), and resolution method (directional high-definition acquisition + algorithm enhancement). The log is associated with a unique identifier for this verification, which is convenient for subsequent traceability.

[0050] This embodiment achieves precise handling of conflicting data sources: it only targets low-weight conflicting data sources, avoiding the inefficiency of full resampling; it balances automation and reliability by limiting the number of collections to N; and it enables the data source performance to be strongly correlated with subsequent weights through event logging and weight updates, thereby achieving self-optimization of the verification system.

[0051] Specifically, the semantic fusion of heterogeneous complementary fields includes: using named entity recognition and date normalization technology to parse the text description in the chip-issued information, mapping and associating the organization name, location, and date entities with the corresponding standard codes in the machine-readable code, and generating an enhanced issuance record that simultaneously contains structured encoding and readable text.

[0052] Specifically, the verification decision factor matrix includes: the verification decision factor matrix is ​​a two-dimensional relational table with key fields as rows and decision attributes as columns, and the decision attributes include: the final fusion value of the field, the original proposal value of each data source, the unique identifier of the adopted data source, the dynamic weight calculation result of the adopted data source in this verification, and the field consistency status marker.

[0053] Specifically, the method for locating the source of the anomaly is as follows: in the verification decision factor matrix, scan the rows of fields marked as "conflict" or "missing" in terms of consistency status, extract all corresponding source data identifiers and weights, and determine the source identifier with the lowest dynamic weight that provides an error or invalid value as the source of the anomaly.

[0054] Specifically, the generation of the handling plan includes: forming a query key by combining the type of the abnormal root source data source with the category of the abnormal field, and matching it in a predefined handling strategy rule base; the handling strategy rule base contains a standardized handling process instruction set for different scenarios.

[0055] In this embodiment, common scenarios and corresponding instruction sets include: 1. Scenarios of abnormal data sources in visual reading The core field (name / date of birth) was misidentified (e.g., due to fuzzy recognition). Real-time operation: Initiate targeted re-sampling of the visual reading module (focus on the abnormal field area, automatic supplemental lighting + high-definition shooting); Verification action: Push the original field image to the manual verification terminal and mark the suspicious area; Data processing: If the re-sampling is successful, update the field value and record the log; if it fails, call the chip / machine-readable source data for cross-verification.

[0056] Physical anti-counterfeiting features were missed. Real-time operation: Trigger the special collection of anti-counterfeiting features in the visual reading module (fluorescent / holographic pattern recognition mode); Verification action: Output the anti-counterfeiting feature comparison list (system preset standard features vs. collected features); Subsequent linkage: If a missed inspection is confirmed, mark the certificate as "pending anti-counterfeiting verification" and synchronize it to the border inspection station.

[0057] 2. Scenarios of abnormal machine-readable data sources Machine-readable code parsing is incomplete (e.g., due to contamination causing missing fields). Real-time operation: Control the machine-readable device to adjust the scanning angle / power and reread; Data processing: Extract the parsed complete fields and associate them with chip data to complete missing items; Verification requirements: Generate a "Machine-readable incomplete - Chip complete" tag for manual verification of correlation.

[0058] The data quality score is low (distortion / abnormal signal-to-noise ratio). Real-time operation: Pause machine-readable primary verification and switch to chip data priority mode; Device linkage: Record the machine-readable device number and push "device calibration reminder" to the technical maintenance system.

[0059] 3. Scenarios of abnormal chip data sources The core fields (nationality / validity period) conflict with other sources. Immediate action: Verify the validity of the chip's digital signature (trace the certificate chain); Verification action: If the signature is valid, mark the source of conflict (such as visual reading) as "low weight" based on the chip data.

[0060] Chip read failed (hardware damage / encryption failure) Immediate Operation: Switch to "Machine Reading + Visual Reading" dual-source verification mode; Verification Action: Trigger secondary verification of document authenticity (manual verification of anti-counterfeiting marks + integrity of machine-readable code); Data Recording: Mark "Chip Failure" and associate the document holder information with the border inspection's key attention list.

[0061] 4. Cross-source field consistency conflict scenarios Biometric (face / fingerprint) mismatch (e.g., visual image vs. chip features) Immediate action: Initiate special biometric data collection (live face detection + fingerprint re-encoding); Verification action: Push "Biometric Comparison Report" (including similarity score) to the duty police officer's terminal; Coordinated handling: If the similarity is lower than the threshold, trigger a second verification of the person's identity (inquire about the issuance information of the certificate).

[0062] The instruction set includes four elements: "real-time technical operation → manual linkage action → data processing rules → subsequent traceability requirements," ensuring that the handling process is standardized and traceable, while also meeting the border inspection's core requirements of "efficiency first + safety as a safety net."

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for verifying entry and exit documents based on dynamic weights and multimodal fusion, characterized in that, include: S1: Obtain visual information from the surface of the document, and extract the document text, face area, and physical anti-counterfeiting features; Read the ID card code, parse the standardized data fields, and score the data quality of the computer-read code; Extract digital biometric features from the chip to verify the validity of the chip's digital signature; The data quality score includes: calculating optical distortion correction parameters by analyzing the edge deformation rate of the machine-readable code image, and calculating the signal-to-noise ratio by comparing the bit error rate of the parsed data, and quantifying the two into a data quality score for the machine-readable data. S2: Construct a verification data graph with the unique identifier of the document as the root node, and mount visually readable, machine-readable, and chip data as attribute nodes; use entity parsing algorithms to perform fuzzy matching based on the core fields of name, date of birth, and nationality, and output multi-source associated aligned data; S3: Based on the multi-source associated alignment data, a dynamic weight allocation model is constructed to perform weighted voting fusion on key fields from the same source; semantic fusion is performed on complementary fields from different sources to generate an issuance record integrating encoding and natural language information; the dynamic weight allocation model includes: dynamic weight coefficients are calculated from the inherent basic weight of the data source and the data quality score weight of the data collected this time; wherein, the preset relationship of the inherent basic weight is: basic weight of chip read data > basic weight of machine read data > basic weight of visual read data; S4: Based on the results of the weighted voting fusion and semantic fusion, a verification decision factor matrix is ​​constructed. The verification decision factor matrix records the final values ​​of key fields, the data sources on which they are based, and their dynamic weights. Based on the consistency status and weights of the fields in the verification decision factor matrix, the verification conclusion is determined. If the verification conclusion is abnormal, the root source data source of the abnormality is located. Based on the location result and the decision path recorded in the verification decision factor matrix, a structured border inspection entry and exit verification record and the corresponding handling plan are generated in combination with the issuance record. The verification decision factor matrix is ​​a two-dimensional relational table with key fields as rows and decision attributes as columns. The decision attributes include: the final fusion value of the field, the original proposal value of each data source, the unique identifier of the adopted data source, the dynamic weight calculation result of the adopted data source in this verification, and the field consistency status marker.

2. The method according to claim 1, characterized in that, The method for verifying the validity of the chip digital signature described in S1 is as follows: a certificate chain is constructed using a pre-set root certificate of the issuing authority, and decryption and hash value comparison operations are performed on the digital signature read from the chip.

3. The method according to claim 1, characterized in that, The verification data graph mentioned in S2 specifically includes: the verification data graph is a directed attribute graph, wherein the unique identifier of the document serves as the central entity node, and visually readable, machine-readable, and chip data serve as feature nodes connected to it through inclusion relationships. Each feature node stores the original data and the extracted feature vector in the form of key-value pairs.

4. The method according to claim 1, characterized in that, The entity parsing algorithm described in S2 is used to perform fuzzy matching based on the core fields of name, date of birth, and nationality. Specifically, it includes: using a composite comparison algorithm based on text similarity and speech similarity, where the text similarity is calculated based on character edit distance and the speech similarity is calculated based on the pinyin or phonetic mapping relationship of double-byte characters; and setting a dynamically adjustable threshold range, which is adjusted based on the current data quality score.

5. The method according to claim 1, characterized in that, The weighted voting fusion of key fields from the same source described in S3 specifically includes: for the same key field, aggregating the values ​​provided by all data sources and their respective dynamic weights, and adopting the value with the highest weight as the fusion result value; if there is a tie for the highest weight, the value provided by the data source with the higher inherent basic weight is adopted first.

6. The method according to claim 5, characterized in that, When merging values ​​from the data source, if the dynamic weight of the conflicting data source is lower than the preset reliability threshold, a re-collection process is triggered, and the conflict event is recorded. The reliability threshold is Y%, where Y is a preset value representing the weight threshold at which the data source is considered basically reliable.

7. The method according to claim 1, characterized in that, The semantic fusion of heterogeneous complementary fields described in S3 specifically includes: using named entity recognition and date normalization technology to parse the text description in the chip-issued information, mapping and associating the organization name, location, and date entities in the chip with the corresponding standard codes in the machine-readable code, and generating an enhanced issuance record that simultaneously contains structured encoding and readable text.

8. The method according to claim 1, characterized in that, The method for locating the root cause of anomalies in S4 is as follows: In the verification decision factor matrix, scan the rows of fields marked with "conflict" or "missing" in the consistency status, extract all corresponding data source identifiers and weights, and determine the source identifier with the lowest dynamic weight that provides an error or invalid value as the root cause of the anomaly.

9. The method according to claim 1, characterized in that, The generation of the handling plan described in S4 specifically includes: forming a query key by combining the type of the abnormal root source data source with the category of the abnormal field, and matching it in a predefined handling strategy rule base; the handling strategy rule base contains a standardized handling process instruction set for different scenarios.

Images