A method and system for identity verification based on multimodal biometrics
By identifying the associated response values and interpretation chains of multimodal biometric data, irreplaceable datasets are selected, solving the static problem of multimodal fusion strategies in existing technologies and improving the reliability and accuracy of identity verification.
Patent Information
- Application Number
- CN202611284867.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-24
- Publication Date
- 2026-11-17
AI Technical Summary
In existing identity verification systems, the fusion strategy for multimodal biometric data cannot be dynamically adjusted according to the real-time acquisition quality, resulting in low-quality modal data interfering with the reliability of identity verification results.
By identifying the correlation response values between biometric data, an interpretation chain and interpretation cluster are established, irreplaceable interpretation datasets are selected, and a verification relationship with identity information is established to output the identity verification results.
It enables structured processing of multimodal biometric data, ensuring that only high-quality data is used for identity verification, thereby improving the reliability and accuracy of verification results.
Smart Images

Figure CN123116246A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to an identity verification method and system based on multimodal biometrics. Background Technology
[0002] Existing identity verification systems are typically deployed at verification terminals and consist of an ID card reading module, a biometric data acquisition module, and an identity verification module. Personnel first use an ID card reader to read document information, and then a camera captures multimodal biometric data, including facial images and liveness images. The system preprocesses, extracts, and encodes each type of data, then performs comprehensive analysis of the multimodal biometric data according to preset rules and compares it against the identity information corresponding to the ID card. Once the identity verification result meets the set conditions, it is transmitted to the verification terminal.
[0003] However, existing technologies typically use pre-defined rules to verify the identity of multimodal biometric data. The proportion of each modality remains relatively constant during the fusion process, which may prevent dynamic adjustments to the fusion strategy based on the real-time acquisition quality of each modality. For example, in a hotel self-check-in scenario, some guests may experience a decrease in facial image quality due to backlighting, wearing masks, or rapid movement, while the liveness detection data remains normal. The system may still use fixed fusion rules to integrate the poor-quality facial features with other modal features, causing low-quality modal data to continuously participate in identity determination. This can interfere with the final identity verification result and reduce its reliability. Summary of the Invention
[0004] The purpose of this invention is to provide an identity verification method and system based on multimodal biometrics, aiming to solve the problems mentioned in the background art.
[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:
[0006] Firstly, an identity verification method based on multimodal biometrics, the method comprising:
[0007] Obtain personnel identity information data and multimodal biometric data to obtain a verification dataset;
[0008] Based on the verification dataset, any biometric data is used as the target data, and the remaining biometric data are used as the reference data. The degree to which the biometric data can be jointly explained by the remaining biometric data is identified, and the associated response value is obtained.
[0009] By selecting biometric data whose correlation response values meet preset correlation values, explanatory relationships are established, and each explanatory relationship is connected into an explanatory chain. The starting point and ending point of the explanatory chain are determined to obtain a set of explanatory chains.
[0010] Based on the interpretation chain set, any biometric data in the interpretation chain is sequentially masked, and the ability of the biometric data to maintain the continuity of the interpretation chain is identified to obtain a stable contribution value.
[0011] By dividing biometric data with explanatory relationships and stable contribution values that meet preset succession values into the same explanatory cluster, and verifying whether the biometric data within each explanatory cluster maintain explanatory relationships, an explanatory cluster set is obtained;
[0012] Based on the interpretation cluster set, each interpretation cluster is removed in turn and the overall interpretation relationship is re-established. It is then identified whether the overall interpretation relationship remains continuous after a certain interpretation cluster is removed, thus obtaining the irreplaceable interpretation dataset.
[0013] Based on the non-substitutable interpretation dataset, establish the verification relationship between each biometric data and identity information, and output the identity verification results.
[0014] Furthermore, based on the verification dataset, any biometric data is used as the target data, and the remaining biometric data are used as reference data. The degree to which the target biometric data can be jointly explained by the remaining biometric data is identified, and the associated response value is obtained, including:
[0015] By calculating the data retention ratio of each target data unit, the degree to which the target data is retained under the combined effect of reference data is identified, and the retention ratio is obtained; by calculating the source coverage ratio of each target data unit, the coverage capability of the target data to be jointly interpreted by multiple reference data is identified, and the source coverage is obtained.
[0016] By calculating the continuous retention ratio of each target data unit, the stability of the target data in the common interpretation process is identified, and the continuous retention term is obtained; by calculating the deviation of the retention ability among each reference data, the distribution balance of the explanatory power of each reference data on the target data is identified, and the source balance term is obtained.
[0017] By integrating the percentage, source coverage, continuity, and source balance terms, the degree to which the target data is jointly explained by other biometric data is identified, and the associated response value is obtained.
[0018] Furthermore, by selecting biometric data whose correlation response values satisfy preset correlation values to establish explanatory relationships, these explanatory relationships are connected into explanatory chains, and the starting and ending points of the explanatory chains are determined, resulting in a set of explanatory chains, including:
[0019] By comparing the correlation response value of each biometric data with the preset correlation value, the biometric data that are greater than the preset correlation value are identified as candidate explanatory data, thus obtaining the candidate explanatory dataset;
[0020] Based on the candidate explanation dataset, explanation relationships are established according to the common explanation objects among the candidate explanation data, and the explanation source and the explained object of each explanation relationship are recorded to obtain the explanation relationship set;
[0021] Based on the set of explanatory relations, the explanatory relations that correspond continuously to the source of explanation and the object being explained are connected sequentially, while keeping the direction of explanation of each explanatory relation consistent, to obtain the initial explanatory chain;
[0022] Based on the initial interpretation chain, identify the interpretation sources that are not the objects being interpreted as the starting point of the interpretation, and identify the objects being interpreted that are not the sources of the interpretation as the ending point of the interpretation, thus obtaining the interpretation chain set.
[0023] Furthermore, based on the interpretation chain set, any biometric data in the interpretation chain is sequentially masked to identify the ability of that biometric data to maintain the continuity of the interpretation chain, thus obtaining a stable contribution value, including:
[0024] Based on the set of interpretation chains, any biometric data in the interpretation chain is sequentially masked, and the number of interpretation relationships that are broken due to masking the biometric data in each interpretation chain is counted. The support capability of the biometric data for the connection structure of the interpretation chain is identified, and the link breakage item is obtained. The ability of the original connection relationship of the interpretation chain to be maintained after masking the biometric data is identified, and the link maintenance item is obtained.
[0025] Based on the number of changes in the starting and ending points of each interpretation chain after the biometric data is masked, the degree of change in the boundary structure of the interpretation chain after masking the biometric data is identified, and the endpoint change term is obtained; the degree of adjustment of the internal structure of the interpretation chain after masking the biometric data is identified, and the position perturbation term is obtained.
[0026] By fusing link breakage, link maintenance, endpoint change, and location disturbance items, and identifying the degree of disturbance in the overall chain structure after masking the biometric data, a stable contribution value is obtained.
[0027] Furthermore, by dividing biometric data with explanatory relationships and stable contribution values that meet preset succession values into the same explanatory cluster, and verifying whether the biometric data within each explanatory cluster maintain explanatory relationships, an explanatory cluster set is obtained, including:
[0028] By comparing the stable contribution value of each biometric data with the preset succession value, biometric data that can maintain a continuous interpretive relationship are identified, and a succession dataset is obtained.
[0029] Based on the continuation dataset, biometric data with continuous interpretation relationships are grouped according to the source of interpretation, the object being interpreted, and the order of interpretation connections among the biometric data. Biometric data in the same continuous interpretation relationship are identified to obtain an initial interpretation cluster.
[0030] Based on the initial interpretation clusters, each biometric data in each initial interpretation cluster is sequentially masked, and the original interpretation relationship between the remaining biometric data is identified to obtain the intra-cluster verification results.
[0031] Based on the intra-cluster validation results, biometric data that retain the original interpretation relationship are retained in the initial interpretation cluster, while biometric data that do not retain the original interpretation relationship are removed from the initial interpretation cluster, resulting in an interpretation cluster set.
[0032] Furthermore, based on the explanatory cluster set, each explanatory cluster is removed sequentially and the overall explanatory relationship is re-established. It is then determined whether the overall explanatory relationship remains continuous after removing a certain explanatory cluster, resulting in an irreplaceable explanatory dataset, including:
[0033] By sequentially selecting any one of the interpretation clusters in the interpretation cluster set as the removal object and removing the removal object from the interpretation cluster set, the interpretation connection relationship between the remaining interpretation clusters is identified, and the removal interpretation cluster set is obtained.
[0034] The overall interpretation relationship is re-established by recognizing the interpretation sources, interpreted objects, and interpretation connection order among the removed interpretation clusters, and it is then identified whether this overall interpretation relationship covers the interpretation connections before removal, thus obtaining the reconstructed interpretation relationship.
[0035] Based on the reconstructed interpretation relationship, the continuity between the overall interpretation relationship and the interpretation connection before removal is compared, and it is identified whether the overall interpretation relationship remains continuous after the object is removed, thus obtaining the continuity identification result.
[0036] Based on the continuous identification results, the objects whose overall interpretation relationship remains continuous are identified as alternative interpretation clusters, and the objects whose overall interpretation relationship does not remain continuous are identified as non-alternative interpretation clusters. The biometric data in each non-alternative interpretation cluster are collected to obtain the non-alternative interpretation dataset.
[0037] Furthermore, based on the non-substitutable interpretation dataset, a verification relationship is established between each biometric data point and identity information, and the identity verification results are output, including:
[0038] By matching each biometric data in the non-substitutable interpretation dataset with identity information, the identity information corresponding to each biometric data is identified, thus obtaining an identity mapping set;
[0039] Based on the identity mapping set, and according to the correspondence between the interpretation relationship of each biometric data in the non-substitutable interpretation dataset and the identity information, the verification relationship between each biometric data is established, and the verification relationship set is obtained.
[0040] Based on the verification relationship set, the consistency of identity information among various biometric data is compared, and it is identified whether each verification relationship points to the same identity information, thus obtaining the verification and identification result;
[0041] Based on the verification results, verification relationships that both point to the same identity information are identified as consistent identity, while verification relationships that do not both point to the same identity information are identified as inconsistent identity, thus obtaining the identity verification result.
[0042] Secondly, an identity verification system based on multimodal biometrics, the system comprising:
[0043] The data acquisition module is used to acquire personnel identity information data and multimodal biometric data to obtain a verification dataset;
[0044] The association module is used to identify the degree to which any biometric data can be jointly explained by the other biometric data, based on the verification dataset, and using any other biometric data as the target data and the remaining biometric data as the reference data, and to obtain the association response value.
[0045] The interpretation chain module is used to establish interpretation relationships by selecting biometric data whose correlation response values meet preset correlation values, connect each interpretation relationship into an interpretation chain, and determine the interpretation start point and interpretation end point to obtain an interpretation chain set;
[0046] The stabilization module is used to sequentially mask any biometric data in the interpretation chain according to the interpretation chain set, identify the ability of the biometric data to maintain the continuity of the interpretation chain, and obtain a stable contribution value.
[0047] The interpretation cluster module is used to divide biometric data with interpretation relationships and stable contribution values that meet preset succession values into the same interpretation cluster, and verify whether the biometric data within the interpretation cluster maintain the interpretation relationship, thereby obtaining an interpretation cluster set;
[0048] The replacement module is used to remove each interpretation cluster in sequence and re-establish the overall interpretation relationship based on the interpretation cluster set, and to identify whether the overall interpretation relationship remains continuous after removing a certain interpretation cluster, thus obtaining the non-replaceable interpretation dataset.
[0049] The verification module is used to establish the verification relationship between various biometric data and identity information based on the non-substitutable interpretation dataset, and output the identity verification results.
[0050] The above-described solution of the present invention has at least the following beneficial effects:
[0051] This invention obtains association response values by identifying the degree to which target biometric data can be jointly explained by other biometric data. This transforms the relationships between multimodal biometric data into response data usable for subsequent processing, enabling heterogeneous multimodal data to form comparable intermediate data results before fusion. The association response values serve as a prerequisite for establishing interpretive relationships, indicating the association status of a particular biometric data point with other biometric data points in the current verification dataset. Before establishing verification relationships, the system preprocesses the inherent connections between multimodal data, ensuring that subsequent operations are not based on the indiscriminate participation of all collected data, but rather on data with associated responses to continue constructing the interpretive structure.
[0052] This invention organizes scattered interpretation relationships into a data structure with a continuous direction by connecting various interpretation relationships into an interpretation chain and determining the interpretation start and end points. At the data processing level, multiple local interpretation relationships are sequentially connected, forming a chain relationship from the interpretation start point to the interpretation end point among multimodal biometric data. The interpretation start point indicates the data position that is not pointed to by other interpretation relationships but can serve as a source of interpretation, and the interpretation end point indicates the data position where the interpretation transmission ends. The system can clearly define the boundaries, directions, and continuous range of the interpretation chain, enabling the assessment of the impact on the continuity of the chain structure when any biometric data is subsequently masked, thus providing a structured data path for the multimodal data fusion process.
[0053] This invention sequentially masks any biometric data in the interpretation chain and identifies the ability of that biometric data to maintain the continuity of the interpretation chain. Without directly relying on manual judgment, it obtains the data function position of each biometric data in the interpretation chain structure. If the interpretation chain breaks, changes its boundaries, or cannot continue to connect after a certain data is masked, then that data has the function of maintaining continuity in the chain structure. If the chain relationship can still be maintained by other data after masking, then that data may be substitutable or non-critical connection data in the current chain. The system can generate a stable contribution value according to the changes in the interpretation chain structure itself, so that the link function of each biometric data is converted into contribution data.
[0054] This invention divides biometric data with interpretable relationships and stable contribution values that meet preset continuity values into the same interpretability cluster. Data with continuous continuity relationships within the interpretability chain are further grouped into cluster structures. Biometric data in the same interpretability chain or capable of maintaining continuous interpretability relationships are organized into the same interpretability cluster. The interpretability cluster serves as a subsequent processing unit, representing a set of data with mutual interpretability and continuous support relationships in the current verification dataset. This allows subsequent removal operations to be performed on a cluster-by-cluster basis, rather than processing only a single modality, thus preserving the combined relationships between multimodal data.
[0055] This invention verifies whether the biometric data within an interpretation cluster maintain their interpretive relationships. It performs internal relationship checks on already formed interpretation clusters, ensuring that interpretation clusters are not solely determined by previous partitioning results, but rather by further examining whether interpretive relationships are still maintained between data within the cluster. If some data within a cluster cannot maintain their original interpretive relationships during verification, that data is no longer considered a stable component of that interpretation cluster; if the data within a cluster maintains their interpretive relationships, the cluster can serve as a processing unit for subsequent removal and reconstruction of the overall interpretive relationships. This establishes a verification mechanism for the consistent connectivity state within clusters at the data processing level, ensuring that each interpretation cluster in the interpretation cluster set has an internal interpretive relationship basis, preventing data lacking sustainable interpretive relationships from being retained in the same cluster due to a single association screening or chain connection. Attached Figure Description
[0056] Figure 1 This is a flowchart of an identity verification method based on multimodal biometrics provided by an embodiment of the present invention.
[0057] Figure 2 This is a flowchart illustrating the process of obtaining associated response values provided in an embodiment of the present invention.
[0058] Figure 3 This is a flowchart illustrating the process of obtaining a stable contribution value, provided by an embodiment of the present invention. Detailed Implementation
[0059] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0060] like Figure 1 As shown, embodiments of the present invention propose an identity verification method based on multimodal biometrics, the method comprising:
[0061] Obtain personnel identity information data and multimodal biometric data to obtain a verification dataset;
[0062] Based on the verification dataset, any biometric data is used as the target data, and the remaining biometric data are used as the reference data. The degree to which the biometric data can be jointly explained by the remaining biometric data is identified, and the associated response value is obtained.
[0063] By selecting biometric data whose correlation response values meet preset correlation values, explanatory relationships are established, and each explanatory relationship is connected into an explanatory chain. The starting point and ending point of the explanatory chain are determined to obtain a set of explanatory chains.
[0064] Based on the interpretation chain set, any biometric data in the interpretation chain is sequentially masked, and the ability of the biometric data to maintain the continuity of the interpretation chain is identified to obtain a stable contribution value.
[0065] By dividing biometric data with explanatory relationships and stable contribution values that meet preset succession values into the same explanatory cluster, and verifying whether the biometric data within each explanatory cluster maintain explanatory relationships, an explanatory cluster set is obtained;
[0066] Based on the interpretation cluster set, each interpretation cluster is removed in turn and the overall interpretation relationship is re-established. It is then identified whether the overall interpretation relationship remains continuous after a certain interpretation cluster is removed, thus obtaining the irreplaceable interpretation dataset.
[0067] Based on the non-substitutable interpretation dataset, establish the verification relationship between each biometric data and identity information, and output the identity verification results.
[0068] In this embodiment of the invention, identity information data and multimodal biometric data of personnel are acquired to obtain a verification dataset. This allows the separately collected identity information data and multimodal biometric data to be included in the same data processing scope, avoiding the lack of unified relationship due to independent processing of different modal data. Based on the verification dataset, any biometric data is used as target data, and the remaining biometric data are used as reference data. The degree to which the biometric data can be jointly explained by the remaining biometric data is identified, and an association response value is obtained. Each type of biometric data can be independently evaluated in the current verification dataset to assess its association status with other modal data, avoiding the pre-fixing of a certain modality as the dominant data. By selecting biometric data whose association response values meet the preset association values, an interpretation relationship is established. Each interpretation relationship is connected into an interpretation chain, and the interpretation start point and interpretation end point are determined to obtain an interpretation chain set, which represents the continuous support relationship between different biometric data. This provides a structural basis for subsequently judging the impact of individual data on the continuity of the overall relationship.
[0069] Based on the interpretation chain set, each biometric data in the interpretation chain is sequentially masked to identify its ability to maintain the continuity of the interpretation chain, thus obtaining a stable contribution value. This provides a processing basis for subsequent interpretation cluster division. By grouping biometric data with interpretation relationships and stable contribution values that meet preset continuity values into the same interpretation cluster, and verifying whether the biometric data within each interpretation cluster maintain the interpretation relationship, an interpretation cluster set is obtained. This cluster set represents stable relationship combinations in multimodal data, enabling subsequent processing to perform alternative identification for data combinations, rather than processing only single-modal data. Based on the interpretation cluster set, each interpretation cluster is sequentially removed and the overall interpretation relationship is re-established. It is then determined whether the overall interpretation relationship remains continuous after removing a certain interpretation cluster, resulting in an irreplaceable interpretation dataset. This dataset identifies which biometric data combinations have an irreplaceable role in the overall interpretation relationship, preventing all collected data from entering the final verification stage indiscriminately. Based on the irreplaceable interpretation dataset, a verification relationship between each biometric data and identity information is established, and the identity verification result is output, ensuring that the verification result has a clear data source and processing path.
[0070] This involves acquiring personnel identity information data and multimodal biometric data to obtain a verification dataset, specifically including:
[0071] After a person enters the identity verification process, the verification terminal receives the verification request triggered by the person and generates a unique verification task identifier for that request. The verification terminal can trigger the verification process through various means, such as a person clicking to confirm, a document approaching the reading area, a camera detecting the person entering the data collection area, an access control device receiving a pass request, or a self-service terminal receiving a business processing request. The verification terminal obtains the person's identity information data through an identity information reading module. This module can be an ID card reader, an electronic certificate reading interface, a document image acquisition device, a mobile terminal certificate reading interface, or a data reading interface connected to an identity information database. When using an ID card reader, the system reads the document number, name, gender, date of birth, document address, issuing authority, document validity period, and document photo from the ID card chip. When using an electronic certificate reading interface, the system reads the identity identifier, document image, document validity status, and identity certificate number from the electronic certificate. When using a document image acquisition method, the system acquires images of both sides of the document and extracts the document number, name, validity period, and document portrait area from the image. The system formats the identity information data, converting text fields such as ID number, name, and ID validity period into a unified encoding format, converting ID photos into a unified image format, binding the identity identifier field to the current verification task identifier, and recording the data reading time, reading device number, and reading source type. For cases with missing fields, abnormal field formats, invalid ID validity periods, or duplicate reading results, the system generates a corresponding data status identifier and stores this identifier along with the identity information data in a temporary data cache.
[0072] The system initiates a multimodal biometric data collection process. Multimodal biometric data can include facial image data, liveness detection data, fingerprint data, iris data, voiceprint data, palm vein data, palmprint data, gait data, or other biometric data that can be used for identity verification. The system determines the types of biometrics to be collected based on the data collection devices configured on the verification terminal, or it can determine at least two biometric modalities as the collection targets for this verification based on the business scenario. For example, in a hotel self-check-in scenario, facial image data and liveness detection data can be collected; in an access control scenario, facial image data, fingerprint data, and palm vein data can be collected; and in a remote identity verification scenario, facial image data, liveness detection data, and voiceprint data can be collected.
[0073] For facial image data, the system acquires facial images or video frames using camera equipment and records data such as acquisition time, camera equipment number, image resolution, frame number, and acquisition angle. Facial image data can include static facial images, continuous video frames, facial region images, and keyframe data extracted from video frames. The system can perform facial region localization on the acquired facial images, marking the area corresponding to the person's face in the image data and using this area as the main part of the facial image data. In cases where no facial region is detected, multiple facial regions are detected, or the facial region does not match the acquisition area, the system can generate a corresponding data status identifier to indicate the acquisition status of the facial image data. For liveness detection data, the system acquires liveness-related data using camera equipment, infrared camera equipment, depth camera equipment, structured light sensors, near-infrared sensors, or motion interaction acquisition methods. The liveness detection data includes blinking action data, mouth opening action data, head nodding or turning action data, facial depth data, infrared reflection data, skin texture response data, and continuous frame change data. When collecting liveness detection data, the system correlates the acquisition time of liveness data with that of facial image data and records the liveness acquisition device number, acquisition frame sequence, action command identifier, and response time information. For fingerprint data, the system acquires fingerprint images or fingerprint feature data of a person's hand through the fingerprint acquisition module, and records the finger position, acquisition time, acquisition device number, and number of acquisitions. The system converts the acquired fingerprint images into a unified image format and marks the fingerprint region, ridge direction, acquisition area, and contact state. For multiple fingerprint data acquired repeatedly, the system can group them into the same modality data group according to the acquisition order and retain the status information of each acquisition. For cases where no effective ridge area is acquired, the contact area is insufficient, or there is duplicate coverage, the system generates a corresponding data status identifier.
[0074] For iris data, the system acquires images of a person's eyes using an iris acquisition device and locates the iris region from these images. During acquisition, the system records the left and right eye identifiers, acquisition distance, acquisition time, acquisition device number, and image format. If multiple eye images are acquired, the system can save the iris region data corresponding to each image separately, treating them as multiple data units within the iris modality. In cases of eye occlusion, unlocated iris regions, abnormal image formats, or acquisition distances outside the set range, the system binds the corresponding data status identifier to the iris data. For voiceprint data, the system acquires a person's voice data using an audio acquisition device and records the voice acquisition time, audio sampling rate, number of channels, acquisition device number, and voice interaction content identifier. The voiceprint data includes voice data generated by a person reading specified content, as well as voice segments generated during business transactions. The system converts the voice data into a unified audio format and binds it to the current verification task identifier. In cases of insufficient voice length, no detected voice segments, interruptions during acquisition, or audio formats not meeting requirements, the system generates a data status identifier and stores it along with the voiceprint data. For palm vein, palm print, or other hand biometric data, the system acquires hand images, near-infrared images, or texture images using appropriate acquisition devices, and records the palm position, acquisition time, acquisition device number, and acquisition method. The system marks the target region in the hand image, enabling subsequent processing to identify the biometric modality to which the data belongs and its acquisition source. In cases where the acquisition area is missing, the target region is not located, or multiple acquisitions of the same modality exist, the system binds the status identifier, acquisition sequence, and data unit number to the modality data.
[0075] The system binds identity information data and multimodal biometric data at the task level, uses the verification task identifier as the main index, takes identity information data as the identity baseline data, takes various types of biometric data as the biometric data to be verified, and establishes an initial correspondence between identity information data and each biometric data to generate a verification dataset.
[0076] In a preferred embodiment of the present invention, based on the verification dataset, any biometric data is used as target data, and the remaining biometric data are used as reference data. The degree to which the biometric data can be jointly explained by the remaining biometric data is identified to obtain an association response value, including:
[0077] By calculating the data retention ratio of each target data unit, the degree to which the target data is retained under the combined effect of reference data is identified, and the retention ratio is obtained; by calculating the source coverage ratio of each target data unit, the coverage capability of the target data to be jointly interpreted by multiple reference data is identified, and the source coverage is obtained.
[0078] By calculating the continuous retention ratio of each target data unit, the stability of the target data in the common interpretation process is identified, and the continuous retention term is obtained; by calculating the deviation of the retention ability among each reference data, the distribution balance of the explanatory power of each reference data on the target data is identified, and the source balance term is obtained.
[0079] By integrating the percentage, source coverage, continuity, and source balance terms, the degree to which the target data is jointly explained by other biometric data is identified, and the associated response value is obtained.
[0080] In this embodiment of the invention, by calculating the data retention ratio of each target data unit, the degree to which the target data is retained under the combined effect of reference data is identified, and a retention ratio item is obtained. The degree to which the target data is jointly retained by the reference data is converted into a data item, avoiding association judgment based solely on the existence of the target data. The source coverage ratio of each target data unit is calculated to identify the coverage capability of the target data jointly interpreted by multiple reference data, and a source coverage item is obtained. This can identify whether the target data is jointly explained by multiple reference modalities, reflecting the joint coverage state between multimodal data. By calculating the continuous retention ratio of each target data unit, the stability of the target data during the joint interpretation process is identified. The degree of continuity is determined by calculating the degree of retention among the reference data to identify the distribution balance of the explanatory power of each reference data on the target data, thus obtaining the source balance term. This term helps to identify the explanatory distribution among the reference data and avoids mistaking the target data for being jointly explained by multiple modalities simply because a particular reference data has a strong explanatory power. The retention percentage, source coverage, continuity, and source balance terms are then fused to identify the degree to which the target data is jointly explained by other biometric data, resulting in an associated response value that reflects the co-explanatory relationship between the target data and other biometric data.
[0081] The formula for calculating the associated response value is as follows:
[0082] ,
[0083] in, For the first The associated response value of a biometric data point For the first The number of target data units that can participate in maintaining identification from the target data. The number of reference data, For the first The number of times a target data unit satisfies the preset retention condition after being restored by the reference data. For the first The number of times a target data unit fails to meet the preset retention conditions after being restored by the reference data. For the first The number of reference data sources that meet the preset retention conditions in each target data unit. For including the first The length of a data unit segment that continuously meets the preset retention conditions and represents a target data unit. The length of the data unit segment containing the u-th target data unit that does not meet the preset retention condition. For the first The number of target data units that meet the preset preservation conditions in the data restoration fragment generated from the reference data. For each reference data The average value.
[0084] Specifically, the target data is divided into multiple target data units according to the data order; data restoration fragments corresponding to the target data are generated according to each reference data; the preset preservation condition is: the reference data and the target data are mapped to a shared feature space of the same preset dimension, for example, 512-dimensional feature vectors are extracted by a pre-trained multimodal deep neural network, and the cosine similarity between the feature vector of the reference data and the feature vector of the target data in the shared feature space is calculated. If the cosine similarity is greater than a preset similarity threshold such as 0.85, it is determined that the preset preservation condition is met.
[0085] Specifically, based on the verification dataset, any biometric data is used as the target data, and the remaining biometric data are used as reference data. The degree to which the target biometric data can be jointly explained by the remaining biometric data is identified, and the associated response value is obtained. This includes:
[0086] For the first The associated response value of the first biometric data, the first One biometric data point is the biometric data currently selected as the target data, while the remaining biometric data points serve as reference data. Indicates the first The system can retain the number of target data units that can participate in the recognition process within a given set of target data. Target data units can be regions in a face image, consecutive video frames, fingerprint ridge fragments, iris texture regions, voiceprint time segments, palm vein image regions, etc. The system divides the target data into multiple target data units according to data order, spatial location, or time window, enabling the target data to be restored and retained unit by unit.
[0087] Indicates the first The system determines the number of times a target data unit satisfies the preset retention condition after being restored by reference data. It generates a data restoration segment corresponding to the target data based on each reference data and determines whether the direction of change of the restoration segment at the same data position is consistent with that of the target data unit. If they are consistent, it is recorded as satisfying the preset retention condition. Indicates the first The number of times a target data unit fails to meet the preset retention conditions after being restored by reference data. The retention ratio of the target data unit under the combined effect of the reference data can be obtained. Adding 1 to the denominator can avoid... and When all values are zero, a division by zero occurs, while ensuring that data units without valid restore records are not mistakenly assigned a high retention ratio.
[0088] Indicates the first The number of reference data sources that meet the preset retention conditions in each target data unit. Indicates the number of reference data. This is used to indicate the source coverage ratio of the target data unit, which is jointly interpreted by multiple reference data. If the... If a target data unit satisfies the preservation condition by only one reference data, then Smaller; if the target data unit can be maintained by multiple reference data simultaneously, then near .
[0089] Indicates containing the first The length of a data unit segment that continuously meets the preset retention conditions and targets a certain number of data units. Indicates containing the first The length of a data unit segment that is a target data unit and does not meet the preset retention conditions. This is used to represent the proportion of the continuous preservation state around a target data unit within the overall continuous segment. For face images, it can be represented by whether adjacent facial regions are continuously preserved by the reference data; for video or liveness data, it can be represented by whether consecutive frames continuously meet the preservation condition; for voiceprint data, it can be represented by whether consecutive speech segments are continuously interpreted. The technical effect of this approach is that it extends the local preservation state of the target data unit to the continuous segment level, so that the associated response value not only considers whether a unit is interpreted, but also whether the interpreted state exists continuously in time, space, or feature order.
[0090] in the formula Indicates the first The retention rate, source coverage, and continuous retention status of all target data units in each target data set are accumulated. Then multiplied by... This indicates that all target data units have undergone averaging. This process ensures that the associated response value of the target data is not directly affected by the number of target data units, avoiding abnormal amplification of the response value due to a large number of units being divided into a certain modality. This part corresponds to the fusion result of the retention ratio, source coverage, and continuity retention terms, used to represent the overall degree to which the target data is jointly interpreted and continuously retained by the reference data at the data unit level.
[0091] Indicates the first The system counts the number of target data units that meet the preset preservation conditions in the data restoration fragment generated from each reference data. For each reference data, the system counts how many target data units in the target data can be successfully preserved or interpreted. Representing each reference data average value. Indicates the first The degree to which the retention capacity of each reference data point deviates from the average retention capacity. If the explanatory power of each reference data point for the target data is relatively similar, then each... and The difference between them is small; if the target data mainly depends on a certain reference data for interpretation, while other reference data contribute less, the deviation is large.
[0092] in the formula Used to indicate the degree of deviation in the ability to maintain consistency between reference data. This means that, theoretically, each reference data point can form a scale benchmark for maintaining identification for all target data units. This represents the total number of target data units that actually meet the preservation conditions. Adding 1 to the denominator is also to avoid the denominator being zero. The greater the deviation, the more unbalanced the interpretability among different reference data. The source balance term is obtained by subtracting the deviation from the outer 1.
[0093] Based on the above calculations, the associated response value Simultaneously, it reflects four types of data processing results: First, the proportion of the target data unit that is preserved by the reference data; second, how many reference data sources cover the target data unit; third, whether the preservation state of the target data unit continues in continuous data segments; and fourth, whether the preservation capabilities of each reference data point for the target data are evenly distributed. These four types of results work together to... Able to represent the first The degree to which a single biometric data point is jointly explained by the remaining biometric data. When When the value is large, it indicates the first... Multiple target data units within a biometric dataset can be preserved by reference data, with a broad range of preservation sources, continuous preservation status, and a relatively balanced distribution of preservation capabilities among the reference data. In this case, a sufficiently comprehensive common interpretative relationship is formed between the target data and other biometric datasets. A smaller value indicates fewer data units that can be retained in the target data, insufficient coverage of reference sources, weak continuous retention, or that the interpretation is mainly based on a single reference data. Therefore, the system can... The comparison result with the preset association value determines whether the biometric data should enter the subsequent interpretation relationship construction process.
[0094] In a preferred embodiment of the present invention, explanatory relationships are established by selecting biometric data whose correlation response values satisfy a preset correlation value, connecting each explanatory relationship into an explanatory chain, and determining the explanatory start point and explanatory end point to obtain an explanatory chain set, including:
[0095] By comparing the correlation response value of each biometric data with the preset correlation value, the biometric data that are greater than the preset correlation value are identified as candidate explanatory data, thus obtaining the candidate explanatory dataset;
[0096] Based on the candidate explanation dataset, explanation relationships are established according to the common explanation objects among the candidate explanation data, and the explanation source and the explained object of each explanation relationship are recorded to obtain the explanation relationship set;
[0097] Based on the set of explanatory relations, the explanatory relations that correspond continuously to the source of explanation and the object being explained are connected sequentially, while keeping the direction of explanation of each explanatory relation consistent, to obtain the initial explanatory chain;
[0098] Based on the initial interpretation chain, identify the interpretation sources that are not the objects being interpreted as the starting point of the interpretation, and identify the objects being interpreted that are not the sources of the interpretation as the ending point of the interpretation, thus obtaining the interpretation chain set.
[0099] In this embodiment of the invention, by comparing the correlation response value of each biometric data with a preset correlation value, biometric data with a correlation value greater than the preset correlation value are identified as candidate explanatory data, thus obtaining a candidate explanatory dataset. This avoids all collected data from participating in the construction of explanatory relationships indiscriminately. Based on the candidate explanatory dataset, explanatory relationships are established according to the common explanatory objects among the candidate explanatory data, and the explanatory source and the explanatory object of each explanatory relationship are recorded, resulting in an explanatory relationship set. This further transforms the original correlation response results between multimodal biometric data into data connection relationships with directional attributes, providing a relational basis for the subsequent construction of continuous explanatory paths. Based on the explanatory relationship set, the explanatory relationships that continuously correspond to the explanatory source and the explanatory object are sequentially connected, while maintaining the consistency of the explanatory direction of each explanatory relationship, to obtain an initial explanatory chain. This expands the local explanatory relationships between data into an overall explanatory structure, providing a chain-like organizational form for the continuous support relationships between data in subsequent analysis. Based on the initial explanatory chain, explanatory sources that are not explanatory objects are identified as explanatory starting points, and explanatory objects that are not explanatory sources are identified as explanatory ending points, resulting in an explanatory chain set, so that each explanatory chain has a clear data start position and end position.
[0100] Specifically, based on the candidate explanatory dataset, explanatory relationships are established according to the common explanatory objects among the candidate explanatory data, and the source of explanation and the object being explained for each explanatory relationship are recorded to obtain the explanatory relationship set, which specifically includes:
[0101] The system reads the modality identifier, data number, association response value, and common interpretation record formed during the association response value calculation of each candidate explanatory data in the candidate explanatory dataset. The common interpretation record indicates which reference data maintain, cover, or continuously interpret a candidate explanatory data when it is used as target data; it also indicates which target data a candidate explanatory data participates in interpreting when it is used as reference data. The system takes any candidate explanatory data in the candidate explanatory dataset as the current object to be explained and queries other biometric data that participated in the common interpretation during the association response value calculation of that candidate explanatory data and also belong to the candidate explanatory dataset. If one or more candidate explanatory data interpret the current object to be explained in the common interpretation record, and the association response value corresponding to the common interpretation record meets a preset association value, then the system determines the one or more candidate explanatory data as the source of explanation for the current object to be explained. The source of explanation is not arbitrarily selected biometric data, but rather data that has already formed a common interpretation for the current object to be explained during the preceding association response value calculation. The preset association value can be obtained by acquiring a multimodal biometric dataset that has been confirmed to belong to the same identity in historical verification tasks, calculating the association response value of each biometric data according to the method of this embodiment, plotting the probability density distribution curve of the association response value, and taking the lower quartile of the distribution curve as the preset association value.
[0102] When an object to be explained is explained by multiple candidate explanatory data, the system can establish multiple explanatory relationships and record multiple explanatory sources and the same object to be explained in the same relationship structure. Specifically, when face image data, liveness detection data, and iris data are all candidate explanatory data, and face image data and liveness detection data jointly affect iris data in the joint explanatory record, the system can establish an explanatory relationship from face image data to iris data, and an explanatory relationship from liveness detection data to iris data; it can also establish a composite explanatory relationship with face image data and liveness detection data as the explanatory sources and iris data as the object to be explained. Regardless of whether a single-source relationship record or a composite relationship record is used, the directionality between the explanatory source and the object to be explained should be preserved. When establishing each explanatory relationship, the system generates a relationship identifier for that relationship and records the explanatory source identifier, the object to be explained identifier, the verification task identifier, the associated response value corresponding to the explanatory relationship, the data unit location corresponding to the explanatory relationship, the modality type of the explanatory source, the modality type of the object to be explained, and the relationship establishment time. The explanation source identifier indicates the candidate explanation data providing explanation support, the explained object identifier indicates the candidate explanation data receiving explanation support, and the data unit position indicates which data units or continuous data segments in the target data the explanation relationship corresponds to. Through these records, the explanation relationship not only indicates a connection between two candidate explanation data sets but also that this connection originates from the previous joint explanation identification result. The system continues to traverse other candidate explanation data in the candidate explanation dataset, repeatedly executing the explanation source query, explained object determination, and explanation relationship establishment processes until every piece of data in the candidate explanation dataset that can serve as an explained object has completed explanation relationship identification. For data not jointly explained by other candidate explanation data, the system does not establish an explanation relationship for it as an explained object but can retain its candidate status as the starting point of subsequent explanation chains; for data that can explain other candidate explanation data but is not explained itself, the system records its explanation source attribute; for data that can be explained by and explain other candidate explanation data, the system records both its explained object attribute and explanation source attribute. After completing the traversal of all candidate explanation data, the system uniformly aggregates all established explanation relationships to form an explanation relationship set.
[0103] Specifically, based on the set of explanatory relations, the explanatory relations that continuously correspond to the source of explanation and the object being explained are sequentially connected, while maintaining the consistent direction of explanation for each relation, to obtain the initial explanatory chain, which includes:
[0104] The system reads the source identifier and the object identifier of each interpretation relation in the interpretation relation set, and establishes a directed relation table according to the interpretation direction. In the directed relation table, each record uses the source identifier as the front-end node and the object being interpreted as the back-end node. The system treats each candidate interpretation data as an interpretation node and each interpretation relation as a directed edge connecting two interpretation nodes, forming a directed interpretation structure composed of candidate interpretation data and interpretation relations. The system selects an interpretation relation that has not yet been connected from the directed relation table as the current relation, determines the source identifier of this current relation as the front-end node of the current link, and determines the object being interpreted as the back-end node of the current link. The system continues to search the interpretation relation set for another interpretation relation whose source identifier is the same as the back-end node of the current link. If it exists, it means that the object being interpreted in the current link can continue to be the source identifier of the next interpretation relation, and the system connects this other interpretation relation to the current link. After the connection is completed, the system updates the object being interpreted in the new connection to the new back-end node of the current link and continues to search for the next consecutive interpretation relation.
[0105] For example, if the set of explanatory relations contains a first explanatory relation A to B and a second explanatory relation B to C, since the object B in the first explanatory relation is the same as the source B in the second explanatory relation, the system will connect the first and second explanatory relations sequentially to form a chain structure from A to B to C. If a third explanatory relation C to D exists, the connection will continue to form from A to B to C to D. During this connection process, the object being explained in the previous relation must correspond continuously to the source of explanation in the next relation, and each connection is made in the direction from the source of explanation to the object being explained. During the connection process, the system maintains the consistency of the explanation direction of each explanatory relation. Specifically, the system only allows the source of explanation in the next explanatory relation to correspond to the object being explained in the previous explanatory relation, and does not allow the object being explained in the next explanatory relation to be used as the connection source of the previous relation, nor does it allow forcibly connecting relations with opposite explanation directions. In the case of multiple connectable relations, the system can select one to continue connecting according to the size of the associated response value, the order of relation establishment, the order of data collection, or the modality priority, or it can form multiple branch initial explanatory chains. For branching structures, the system records links connecting the same front-end node to different back-end nodes separately to ensure that each initial interpretation chain has a single, clear interpretation direction. When the system cannot find a next interpretation relationship with that back-end node as the source of interpretation at the current link's back-end node, it indicates that the current link cannot continue to connect. The system terminates the connection process of this link and records it as an initial interpretation chain. The system selects another interpretation relationship from the set of interpretation relationships that has not yet formed an initial interpretation chain and repeats the above connection process until all interpretation relationships in the set of interpretation relationships have been traversed. For cases containing only a single interpretation relationship that cannot be connected further, the system also records this single interpretation relationship as an initial interpretation chain. The system generates a link identifier for each initial interpretation chain. If the same interpretation relationship has already been used by an initial interpretation chain, the system can mark its connected status according to preset rules. If branch links are allowed to share the same front-end relationship, the system records multiple link identifiers to which the interpretation relationship belongs to ensure that the shared parts between different initial interpretation chains can be identified later.
[0106] In a preferred embodiment of the present invention, based on the interpretation chain set, any biometric data in the interpretation chain is sequentially masked, and the ability of the biometric data to maintain the continuity of the interpretation chain is identified to obtain a stable contribution value, including:
[0107] Based on the set of interpretation chains, any biometric data in the interpretation chain is sequentially masked, and the number of interpretation relationships that are broken due to masking the biometric data in each interpretation chain is counted. The support capability of the biometric data for the connection structure of the interpretation chain is identified, and the link breakage item is obtained. The ability of the original connection relationship of the interpretation chain to be maintained after masking the biometric data is identified, and the link maintenance item is obtained.
[0108] Based on the number of changes in the starting and ending points of each interpretation chain after the biometric data is masked, the degree of change in the boundary structure of the interpretation chain after masking the biometric data is identified, and the endpoint change term is obtained; the degree of adjustment of the internal structure of the interpretation chain after masking the biometric data is identified, and the position perturbation term is obtained.
[0109] By fusing link breakage, link maintenance, endpoint change, and location disturbance items, and identifying the degree of disturbance in the overall chain structure after masking the biometric data, a stable contribution value is obtained.
[0110] In this embodiment of the invention, based on the interpretation chain set, any biometric data in the interpretation chain is sequentially masked, and the number of interpretation relationships broken due to masking the biometric data in each interpretation chain is counted. The support capability of the biometric data for the connection structure of the interpretation chain is identified, resulting in a link breakage term, reflecting the impact of the masked object on the local connection of the link. The ability to maintain the original connection relationship of the interpretation chain after masking the biometric data is identified, resulting in a link preservation term, representing the degree to which the original connection relationship in the interpretation chain is preserved. Based on the number of changes in the interpretation start point and interpretation end point in each interpretation chain after masking the biometric data, the degree of change in the boundary structure of the interpretation chain after masking the biometric data is identified, resulting in an endpoint change term, representing the change in the boundary structure of the interpretation chain after masking a certain biometric data. The degree of adjustment of the internal structure of the interpretation chain after masking the biometric data is identified, resulting in a position disturbance term, representing the degree of adjustment of the internal structure of the interpretation chain after the masking operation. The link breakage term, link preservation term, endpoint change term, and position disturbance term are fused to identify the degree of disturbance of the overall structure of the interpretation chain after masking the biometric data, resulting in a stable contribution value, representing the degree of role of the biometric data in maintaining the continuous structure of the interpretation chain.
[0111] The formula for calculating the stable contribution value is as follows:
[0112] ,
[0113] in, For the first Stable contribution value of each biometric data point To explain why the chain set contains the first The number of interpretation chains for each biometric data point For including the first The first biometric data The number of biometric data in the interpretation chain, To block the After the first biometric data, the second The number of broken interpretation relations in an interpretation chain. To block the After the first biometric data, the second The number of explanatory relationships is still maintained in each chain of explanation. To block the After the first biometric data, the second The number of times the starting and ending points of an explanation chain have changed. To block the After the first biometric data, the second The total change in the arrangement of the remaining biometric data in the interpretation chain relative to the original interpretation chain.
[0114] Specifically, based on the interpretation chain set, any biometric data in the interpretation chain is sequentially masked to identify the ability of that biometric data to maintain the continuity of the interpretation chain, thus obtaining a stable contribution value. This includes:
[0115] Indicates the first The stable contribution value of each biometric data point. Each biometric data point is a biometric data point selected sequentially from the interpretation chain set and subjected to masking processing. The stability contribution value is used to indicate the degree to which the biometric data point contributes to maintaining the continuity of the link structure in the interpretation chain. The interpretation chain set contains the first... The number of interpretation chains for a given biometric data point, if the first... If a biometric data point appears in multiple interpretation chains simultaneously, the system needs to perform masking identification in each interpretation chain containing the data and average the perturbation results corresponding to each interpretation chain.
[0116] Indicates containing the first The first biometric data The number of biometric data points in each interpretation chain represents the chain size of the t-th interpretation chain. For example, if an interpretation chain goes from face image data to liveness detection data to iris data to voiceprint data, then this interpretation chain contains 4 biometric data points. The value is 4. It is not only used to represent link length, but also serves as a normalization basis in the calculation of link breakage, link maintenance, and position disturbance, enabling disturbance results in interpretation chains of different lengths to participate in the calculation of stability contribution values on the same scale. Indicates blocking the first After the first biometric data, the second The number of broken interpretation relationships in the interpretation chain, the system masks the first one. Before reading the first biometric data, first read the first... All explanatory relations and their order of connection in the chain of explanation; when the first When a biometric data point is masked, any interpretation relationship that uses that biometric data as the source or object of interpretation will be unable to maintain its original connection. The system will count the number of these disconnected interpretation relationships. If the masked data is located in the middle of the interpretation chain, it usually causes the interpretation relationship between its preceding and following sides to be broken; if it is located at the beginning or end of the interpretation chain, it usually causes the interpretation relationship on one side to be broken. (In the formula...) This is used to convert link breakage terms into proportional data related to the explained link length. The larger the value, the more interpretation relationships are broken after the biometric data is masked, and the more obvious the local damage to the continuous structure of the interpretation chain. The larger the value, the more data nodes the interpretation chain contains. The relative impact of the same number of break relationships in a long chain is constrained by the chain size. Adding 1 to both the numerator and denominator can prevent abnormal proportions when the number of break relationships is zero or the chain size is small.
[0117] Indicates blocking the first After the first biometric data, the second The number of interpretation relationships that are still maintained in the original interpretation chain is determined by the system after masking. The system checks if the original interpretation chain does not involve the first interpretation relationship. Whether the interpretive relationships of biometric data can still be maintained in their original direction and order. For interpretive relationships that can still be maintained, include... If most interpretation relationships are preserved after masking, it indicates that the biometric data has a relatively small disruptive effect on the overall interpretation chain; if only a small number of interpretation relationships are preserved after masking, it indicates that the data plays a strong role in the chain's continuity. (In the formula...) Used to reverse the link preservation item. Since the stable contribution value represents the supporting role of the masked data in interpreting the continuous structure of the link, the more interpretation relationships are maintained after masking, the lower the indispensability of the masked data; the fewer interpretation relationships are maintained after masking, the more significant the role of the masked data in link continuity.
[0118] Indicates blocking the first After the first biometric data, the second The system records the number of times the starting and ending points of the interpretation chain have changed before masking. The original starting and ending points of the interpretation chain are defined. After masking, the system re-identifies the starting and ending points of the interpretation chain or sub-chain based on the remaining interpretation relationships. If the original starting point is masked, or a new starting point is generated after masking, it is recorded as an endpoint change; if the original ending point is masked, or a new ending point is generated after masking, it is also recorded as an endpoint change. (In the formula...) To form endpoint change items, an interpretation chain typically has two boundary endpoints: the interpretation start point and the interpretation end point. Therefore, using 2 as the normalization benchmark can represent the proportion of endpoint changes to the boundary structure. If A value of 0 indicates that the interpretation chain boundary did not change after masking the biometric data; a value of 1 for this item means that the aforementioned perturbation results are not amplified. If the value is 1 or 2, it indicates that one or two endpoints have changed, and the value increases accordingly.
[0119] Indicates blocking the first After the first biometric data, the second The total change in the arrangement of remaining biometric data in the interpretation chain relative to the original interpretation chain before shielding. In specific implementation, the system records the first... The system interprets the sequence number and adjacency relationship of each biometric data in the chain; after masking, the system redetermines the sequence number of the remaining biometric data in the remaining links or sub-chains, and compares the position of each remaining biometric data after masking with its position before masking. The sum of the position changes of each remaining biometric data is then obtained. If the remaining data largely maintains its original order after being masked, then Smaller; if the link is split, nodes are rearranged, or the links to which multiple nodes belong change after shielding, then Larger. In the formula Used to normalize the position disturbance term into a disturbance amplification factor. Used to indicate the first This explains the scale benchmark for possible arrangement changes among nodes in the chain. Adding 1 to the denominator avoids division by zero when the chain length is short. If... If the value is small, the term is close to 1, indicating that the shielded object has little impact on the internal arrangement structure of the chain; if A larger value indicates that the masked object causes a more significant change in the position of the remaining data within the chain. The technical advantage of this value is that it allows the stability contribution value to account for changes in the order of nodes within the chain and link rearrangements, rather than simply counting the number of breaks or holds.
[0120] when A larger value indicates that the first [number] is masked. After a number of biometric data points, the interpretation chain is broken in many ways, with fewer original relationships remaining. The starting or ending points of the interpretation change, and the arrangement of the remaining data is significantly disturbed. At this point, the... Individual biometric data plays a strong role in the continuity of interpretation. When When the value is small, it means that after the biometric data is masked, most relationships in the interpretation chain can still be maintained, the endpoints change less, and the arrangement structure within the chain changes less, indicating that the data has a weak continuation effect in the current interpretation chain structure.
[0121] In a preferred embodiment of the present invention, biometric data with interpretable relationships and stable contribution values satisfying a preset succession value are divided into the same interpretable cluster, and the interpretable cluster set is obtained by verifying whether the biometric data within each interpretable cluster maintains the interpretable relationship. The set includes:
[0122] By comparing the stable contribution value of each biometric data with the preset succession value, biometric data that can maintain a continuous interpretive relationship are identified, and a succession dataset is obtained.
[0123] Based on the continuation dataset, biometric data with continuous interpretation relationships are grouped according to the source of interpretation, the object being interpreted, and the order of interpretation connections among the biometric data. Biometric data in the same continuous interpretation relationship are identified to obtain an initial interpretation cluster.
[0124] Based on the initial interpretation clusters, each biometric data in each initial interpretation cluster is sequentially masked, and the original interpretation relationship between the remaining biometric data is identified to obtain the intra-cluster verification results.
[0125] Based on the intra-cluster validation results, biometric data that retain the original interpretation relationship are retained in the initial interpretation cluster, while biometric data that do not retain the original interpretation relationship are removed from the initial interpretation cluster, resulting in an interpretation cluster set.
[0126] In this embodiment of the invention, by comparing the stable contribution value of each biometric data with a preset continuation value, biometric data that can maintain a continuous explanatory relationship are identified, resulting in a continuation dataset. The stable contribution value is then transformed into an explanatory cluster partitioning condition, allowing explanatory clusters to form based on the continuous structural effects within the explanatory chain. Based on the continuation dataset, biometric data with a continuous explanatory relationship are grouped according to the explanatory source, the object being explained, and the order of explanatory connections among the biometric data. Biometric data within the same continuous explanatory relationship are identified, resulting in an initial explanatory cluster set. This ensures that the explanatory clusters have a clear link source and connection basis, providing a foundation for subsequent intra-cluster solutions. Interpretive relationship verification provides the basic structure; based on the initial interpretation cluster set, any biometric data in each initial interpretation cluster is sequentially masked, and it is identified whether the remaining biometric data still maintain the original interpretation relationship, thus obtaining the intra-cluster verification result. Data in the initial interpretation cluster that was included due to link aggregation but cannot stably maintain the intra-cluster interpretation relationship is identified, providing a verification basis for the interpretation cluster set; based on the intra-cluster verification result, biometric data that still maintain the original interpretation relationship are retained in the initial interpretation cluster, and biometric data that do not maintain the original interpretation relationship are removed from the initial interpretation cluster, thus obtaining the interpretation cluster set, enabling subsequent alternative identification to use structurally stable data combinations as processing objects.
[0127] Specifically, by comparing the stable contribution value of each biometric data with a preset succession value, biometric data that can maintain a continuous interpretable relationship are identified, resulting in a succession dataset, which includes:
[0128] The system reads a preset continuation value, which can be obtained by acquiring the median of the stable contribution values of biometric data that have been confirmed to maintain the continuity of the interpretation chain in historical verification tasks. This preset continuation value can be pre-stored in the parameter configuration table of the verification system, or it can be configured according to different verification scenarios, different modal combinations, or different security levels. Under the same verification task identifier, the system compares the stable contribution value of each biometric data with the preset continuation value in turn. When the stable contribution value of a biometric data is greater than or equal to the preset continuation value, the system marks the biometric data as continuation data and records that it can participate in the continuous interpretation relationship to maintain identification; when the stable contribution value of a biometric data is less than the preset continuation value, the system does not use it as continuation data to participate in the current interpretation cluster division. When the same biometric data appears in multiple interpretation chains at the same time, the system can read the comprehensive stable contribution value of the biometric data, or it can read its stable contribution sub-values in each interpretation chain, and when its stable contribution sub-value in at least one interpretation chain meets the preset continuation value, it marks its link position in the corresponding interpretation chain as the continuation position. After comparing all biometric data, the system generates a continuation dataset. This continuation dataset includes at least the data ID, modality identifier, stable contribution value, interpretation chain ID satisfying the continuation conditions, link location ID, corresponding interpretation source, corresponding interpreted object, and verification task identifier. Data that does not meet the preset continuation values is recorded as non-continuation data, retaining its original interpretation relationship, but is not used as the primary object for subsequent initial interpretation cluster aggregation.
[0129] Specifically, based on the continuation dataset, biometric data with continuous explanatory relationships are grouped according to the source of interpretation, the object being interpreted, and the order of interpretation connections among the biometric data. Biometric data within the same continuous explanatory relationship are then identified to obtain an initial explanatory cluster set, which specifically includes:
[0130] The system uses the interpretation chain number as an index to group consecutive data belonging to the same interpretation chain into the same set of pending links. Then, it sorts the consecutive data in the set of pending links according to the link position number or the interpretation connection order, restoring the sequential connection relationship of each consecutive data in the original interpretation chain. After sorting, the system traverses the consecutive data sequentially from the interpretation start point to the interpretation end point of the interpretation chain. During traversal, the system determines whether a continuous interpretation relationship exists between the current consecutive data and its successor. This continuous interpretation relationship requires at least that the current consecutive data is the interpretation source and the successor is the interpretation object, or that the interpretation object corresponding to the current consecutive data is continuously connected to the interpretation source corresponding to the successor; simultaneously, their connection order in the interpretation chain should be adjacent or they should be able to connect to each other through a continuous interpretation relationship. If the current consecutive data and the successor satisfy a continuous correspondence of interpretation source, interpretation object, and interpretation connection order, the system will group them into the same initial interpretation cluster as candidates. If there is no interpretation relationship between the two, the interpretation directions are inconsistent, the connection order is interrupted, or although both are continuation data, they belong to different discontinuous interpretation segments, the system will terminate the current collection at that point and continue collection with the next continuation data as the new starting data.
[0131] When an interpretation chain contains multiple consecutive segments, the system generates multiple initial interpretation clusters. For example, if an interpretation chain contains five biometric data points A, B, C, D, and E, where A, B, and C have consecutive interpretation relationships and are all consecutive data points, D is not consecutive data point, and E, although consecutive data point, cannot be continuously connected to C due to the interruption of D, the system will group A, B, and C into one initial interpretation cluster, without forcibly including E in that cluster. If E can continue to form consecutive interpretation relationships with other subsequent consecutive data points, the system will establish a new initial interpretation cluster starting from E. When multiple interpretation chains contain the same combination of consecutive data points, the system can determine whether they belong to a duplicate cluster based on the consecutive data point number, interpretation relationship number, and connection order. If they are the same, the records are merged; if the connection order or interpretation direction is different, they are saved as different initial interpretation clusters. The system generates a cluster identifier for each initial interpretation cluster and records the biometric data number within the cluster, the modality identifier of each data, the source of interpretation, the object being interpreted, the connection order, the number of the interpretation chain to which it belongs, the cluster start data, the cluster end data, and the stable contribution value on which the cluster was formed.
[0132] Specifically, based on the initial interpretation cluster set, any biometric data in each initial interpretation cluster is sequentially masked, and the remaining biometric data are identified to determine whether the original interpretation relationship is still maintained, thus obtaining the intra-cluster verification result, which includes:
[0133] The system sequentially selects an initial interpretation cluster as the current verification cluster and reads the arrangement order, interpretation source, interpreted object, and intra-cluster interpretation relationships of each biometric data in the initial interpretation cluster. The system saves the original intra-cluster structure of the current verification cluster, which includes the number of intra-cluster data, the number of intra-cluster interpretation relationships, the cluster start data, the cluster end data, the interpretation direction between each biometric data, and the connection order of each interpretation relationship. After saving the original structure, the system begins to perform masking verification on each biometric data in the initial interpretation cluster. In one masking verification, the system selects any biometric data in the current verification cluster as the intra-cluster masking object and temporarily removes the intra-cluster masking object and its directly associated interpretation relationships in this round of verification. Subsequently, the system uses the remaining biometric data as the processing object and re-examines whether the remaining data can still maintain the interpretation relationship according to the interpretation direction and connection order in the original intra-cluster structure. If the remaining biometric data still have the same interpretation source and interpreted object correspondence as the original intra-cluster structure, and can form a continuous interpretation path continuing from the original cluster start direction to the original cluster end direction, then the system determines that the remaining biometric data maintain the original interpretation relationship after this round of masking. If the interpretation direction is reversed, the interpretation source and the interpreted object do not correspond, the connection order is interrupted, the original interpretation path is broken, or the remaining data cannot form a continuous interpretation path, the system determines that the original interpretation relationship was not maintained after this round of masking.
[0134] When the masked object is located at the beginning of the cluster, the system focuses on checking whether the remaining data can use the original second data as the new starting data and maintain the subsequent interpretation direction. When the masked object is located at the end of the cluster, the system focuses on checking whether the preceding remaining data can still maintain the original interpretation relationship. When the masked object is located in the middle of the cluster, the system focuses on checking whether there is a connectable interpretation relationship between the data before and after the masked object, or whether the remaining data can form a continuous path through the original interpretation relationship. The system will not treat relationships with opposite interpretation directions as maintenance relationships, nor will it directly replace the original interpretation relationship with temporary relationships that do not belong to the original cluster structure, unless the replacement relationship already exists in the interpretation relationship set and conforms to the original connection direction. The system performs a masking verification once for each biometric data in the current verification cluster and records the verification results for each round. The intra-cluster verification results may include the cluster identifier, the masked data number, the remaining data number after masking, the original interpretation relationship maintenance identifier, the number of maintained interpretation relationships, the number of broken interpretation relationships, whether the interpretation direction is consistent, whether the connection order is continuous, and whether a continuous interpretation path is formed. After completing all mask verifications for the current verification cluster, the system summarizes the verification results of each round to obtain the intra-cluster verification results for the initial interpretation cluster. Then, the same process is performed on the next initial interpretation cluster until all initial interpretation clusters in the initial interpretation cluster set have been verified.
[0135] In a preferred embodiment of the present invention, based on the interpretation cluster set, each interpretation cluster is sequentially removed and the overall interpretation relationship is re-established. The continuity of the overall interpretation relationship is then determined after removing a certain interpretation cluster, resulting in an irreplaceable interpretation dataset, including:
[0136] By sequentially selecting any one of the interpretation clusters in the interpretation cluster set as the removal object and removing the removal object from the interpretation cluster set, the interpretation connection relationship between the remaining interpretation clusters is identified, and the removal interpretation cluster set is obtained.
[0137] The overall interpretation relationship is re-established by recognizing the interpretation sources, interpreted objects, and interpretation connection order among the removed interpretation clusters, and it is then identified whether this overall interpretation relationship covers the interpretation connections before removal, thus obtaining the reconstructed interpretation relationship.
[0138] Based on the reconstructed interpretation relationship, the continuity between the overall interpretation relationship and the interpretation connection before removal is compared, and it is identified whether the overall interpretation relationship remains continuous after the object is removed, thus obtaining the continuity identification result.
[0139] Based on the continuous identification results, the objects whose overall interpretation relationship remains continuous are identified as alternative interpretation clusters, and the objects whose overall interpretation relationship does not remain continuous are identified as non-alternative interpretation clusters. The biometric data in each non-alternative interpretation cluster are collected to obtain the non-alternative interpretation dataset.
[0140] In this embodiment of the invention, by sequentially selecting any one of the explanatory clusters in the explanatory cluster set as the removal target and removing the removal target from the explanatory cluster set, the explanatory connection relationship between the remaining explanatory clusters is identified to obtain the removed explanatory cluster set. Each explanatory cluster is treated as an independent data combination for substitution identification. An overall structural impact analysis is performed on the explanatory clusters formed after previous verification. The overall explanatory relationship is re-established through the explanatory source, the explanated object, and the explanatory connection order between each removed explanatory cluster, and it is identified whether the overall explanatory relationship covers the explanatory connections before removal, to obtain the reconstructed explanatory relationship. The substitution of the explanatory clusters is transformed into a verifiable overall explanatory relationship reconstruction result, providing a data foundation for subsequent continuous identification. Based on the reconstruction... The interpretation relationship is compared with the continuity of the interpretation connection before removal. It is determined whether the overall interpretation relationship remains continuous after the removal of the object, and the continuity identification result is obtained. This can determine whether the removal of the object causes an irreversible break in the overall interpretation structure. Based on the continuity identification result, the removed objects whose overall interpretation relationship remains continuous are identified as replaceable interpretation clusters, and the removed objects whose overall interpretation relationship does not remain continuous are identified as non-replaceable interpretation clusters. The biometric data in each non-replaceable interpretation cluster is collected to obtain the non-replaceable interpretation dataset. The data combination necessary to maintain the continuity of the overall interpretation relationship is selected from the interpretation cluster set as the input for the subsequent establishment of the verification relationship between biometric data and identity information.
[0141] Specifically, by sequentially selecting any one interpretation cluster from the interpretation cluster set as the removal target and removing the removal target from the interpretation cluster set, the interpretation connection relationships between the remaining interpretation clusters are identified to obtain the removal interpretation cluster set, which specifically includes:
[0142] The system reads the overall interpretation connection structure composed of all interpretation clusters before removal. This overall interpretation connection structure records the connection direction between interpretation clusters, the inter-cluster connection order, and the arrangement position of each interpretation cluster in the overall interpretation structure when no interpretation clusters have been removed. The system uses this overall interpretation connection structure as the baseline structure and retains it in each subsequent round of removal processing for comparison with the reconstruction results after removal. The system selects any interpretation cluster in the interpretation cluster set as the removal target according to a preset removal order. The preset removal order can be the interpretation cluster identification order, the arrangement order in the interpretation chain, the order of appearance of the cluster's initial data, or the interpretation cluster generation order. When an interpretation cluster is selected as the removal target, the system temporarily removes the interpretation cluster from the interpretation cluster set in the current removal round. The temporary removal means that the interpretation cluster, its intra-cluster biometric data, intra-cluster interpretation relationships, and inter-cluster interpretation relationships directly connected to the interpretation cluster do not participate in the overall interpretation relationship reconstruction in the current round, but the original data is still stored in the interpretation cluster set and will be restored to participate in the processing when other interpretation clusters are removed in the next round.
[0143] After the object to be removed is taken, the system identifies the remaining interpretation clusters as candidate interpretation clusters to be removed and reads the cluster start data, cluster end data, interpretation source, interpreted object, and connection order of these remaining interpretation clusters. The system identifies whether there are connectable interpretation relationships between the remaining interpretation clusters. Specifically, if the cluster end data or the interpreted object at the end of a certain remaining interpretation cluster can form a continuous correspondence with the cluster start data or the interpretation source at the beginning of another remaining interpretation cluster, and the interpretation direction between the two is consistent with the direction in the overall interpretation connection structure before removal, then the system establishes a post-removal inter-cluster interpretation connection relationship between the two remaining interpretation clusters. If there is no correspondence between the interpretation source and the interpreted object between the two remaining interpretation clusters, or if there is a data correspondence but the interpretation direction is inconsistent with that before removal, then no inter-cluster interpretation connection relationship is established. For cases where the object to be removed is located in the middle of the overall interpretation structure, the system focuses on checking whether there is a connectable interpretation connection relationship between the interpretation cluster before and after the object to be removed. If the terminating data of the previous interpretation cluster can continue to serve as the source of the starting data for the subsequent interpretation cluster, or if there is a consistent directional interpretation relationship between the two that has been recorded in the interpretation relationship set, the system will establish a connection between them after removal. If the preceding and following interpretation clusters cannot be connected, a break will form between the removed interpretation clusters at that position. For cases where the removed object is located at the beginning or end of the overall interpretation structure, the system checks whether the remaining interpretation clusters can form new starting or ending connections and records the corresponding boundary change information. The system performs connection identification on all remaining interpretation clusters after the removal of the object and records the interpretation clusters that can form connections, the interpretation clusters that cannot form connections, and the connection break positions, generating a set of removed interpretation clusters.
[0144] Specifically, the overall interpretation relationship is re-established by considering the interpretation sources, interpreted objects, and interpretation connection order among the removed interpretation clusters. It is then identified whether this overall interpretation relationship covers the interpretation connections before removal, thus obtaining the reconstructed interpretation relationship. This process includes:
[0145] The system uses the remaining interpretation clusters in the removed interpretation cluster set as reconstruction nodes and the identified inter-cluster interpretation connections between the remaining interpretation clusters as reconstruction edges to re-establish the overall interpretation relationship after removal. The system identifies the reconstruction starting cluster from the removed interpretation cluster set. The reconstruction starting cluster is an interpretation cluster among the remaining interpretation clusters after removal that is not pointed to by other remaining interpretation clusters as the object of interpretation but can continue to point to other interpretation clusters as an interpretation source. If multiple reconstruction starting clusters exist, the system generates different candidate reconstruction paths starting from each reconstruction starting cluster. After determining the reconstruction starting cluster, the system sequentially searches for the next interpretation cluster that can be continuously connected to the current interpretation cluster, according to the direction from the interpretation source to the object of interpretation. If the cluster termination data of the current interpretation cluster and the cluster start data of the next interpretation cluster have a correspondence between interpretation source and object of interpretation, and their connection order conforms to the overall interpretation connection direction before removal, the system connects the next interpretation cluster to the current reconstruction path. The system continues to use the next interpretation cluster as the current interpretation cluster and searches for interpretation clusters that can be continuously connected until no next interpretation cluster that meets the connection conditions can be found. The system maintains a consistent interpretation direction throughout the process, does not make reverse connections between the interpretation source and the interpreted object, and does not use connections that do not exist in the interpretation relation set or the set of removed interpretation clusters as the basis for reconstruction.
[0146] When reconstructing the overall interpretation relationship, the system simultaneously retains the verified intra-cluster interpretation relationships within each interpretation cluster, as well as the inter-cluster interpretation connections between different interpretation clusters. The reconstructed interpretation relationship includes both the interpretation direction and connection order between biometric data within a cluster, and the overall connection path between interpretation clusters. If the remaining interpretation clusters can form a continuous path, the system records this path as a reconstructed overall interpretation relationship; if the remaining interpretation clusters form multiple discontinuous paths, the system records them as multiple reconstructed sub-relationships and marks the overall reconstruction as having a break. The system identifies whether the overall interpretation relationship covers the interpretation connections before removal, reads the remaining baseline interpretation connections in the overall interpretation connection structure before removal (excluding the removed object and its directly associated connections), and uses these as the coverage comparison objects. The system compares the intra-cluster and inter-cluster interpretation relationships in the reconstructed overall interpretation relationship with the remaining baseline interpretation connections one by one. If a pre-removal interpretation connection still exists in the reconstructed overall interpretation relationship, or can be covered by a continuous interpretation path in the same interpretation direction after reconstruction, the system marks the interpretation connection as covered. If a pre-removal interpretation connection does not exist in the reconstructed overall interpretation relationship and cannot be covered by a continuous path between the remaining interpretation clusters, it is marked as uncovered. After completing the coverage identification, the system generates the reconstructed interpretation relationship. The reconstructed interpretation relationship includes at least the current removal object identifier, the reconstruction starting cluster, the reconstruction ending cluster, the reconstruction path, the covered interpretation connection, the uncovered interpretation connection, the number of reconstructed sub-relationships, the reconstruction direction identifier, the reconstruction coverage ratio, and the identifier of the verification task to which it belongs. If the covered interpretation connection can continuously cover the main interpretation connections before removal (excluding the removed object), it indicates that the remaining interpretation clusters after removal can reconstruct the overall interpretation relationship; if there are multiple uncovered connections or the reconstruction path is broken, it indicates that there is a gap in the overall interpretation relationship after removal.
[0147] Specifically, based on the reconstructed interpretation relationship, the continuity between the overall interpretation relationship and the interpretation connection before removal is compared to identify whether the overall interpretation relationship remains continuous after the removal of the object, thus obtaining a continuity identification result, which includes:
[0148] The system performs a continuity comparison between the reconstructed interpretation relationship and the overall interpretation connection structure before removal. The system reads the interpretation direction, interpretation start point, interpretation end point, interpretation cluster arrangement order, inter-cluster connection order, and baseline interpretation connection set from the overall interpretation connection structure before removal; simultaneously, it reads the reconstruction start cluster, reconstruction end cluster, reconstruction path, covered interpretation connections, uncovered interpretation connections, and the number of reconstructed sub-relationships from the reconstructed interpretation relationship. The system uses the overall interpretation connection structure before removal as the continuity comparison benchmark and the reconstructed interpretation relationship after removal as the comparison object. The system determines whether the interpretation direction of the reconstructed interpretation relationship is consistent with the overall interpretation connection structure before removal. If the reconstruction path contains a reversed interpretation direction, an interchange between the interpretation source and the interpreted object, or uses a relationship inconsistent with the original direction to maintain connection, the system directly determines that the reconstructed interpretation relationship does not meet the continuity condition. If all connections in the reconstructed path are established along the original interpretation direction, the system continues to compare whether the reconstructed path can continue along the direction from the interpretation start point to the interpretation end point before removal.
[0149] The system determines whether a continuous path exists in the reconstructed interpretation relationship. It checks whether adjacent interpretation clusters within the reconstructed path maintain a continuous correspondence between the interpretation source and the interpreted object, checks for any gaps that cannot be connected, and checks for the formation of multiple unconnected reconstructed sub-relationships. If the reconstructed interpretation relationship forms only one continuous path, and this path covers all continuous interpretation connections before removal (excluding the removed object), the system determines that the overall interpretation relationship remains continuous after the object is removed. If the reconstructed interpretation relationship is split into multiple sub-paths, has intermediate breakpoints, has uncovered critical interpretation connections, or the starting and ending reconstruction clusters cannot correspond to the remaining boundaries in the structure before removal, the system determines that the overall interpretation relationship is not continuous. For cases where the removed object is located in the middle of the overall interpretation structure, the system also compares whether the interpretation clusters on either side of the removed object can be connected in the reconstructed relationship. If the preceding and following interpretation clusters can continue to connect through existing interpretation connections between the remaining interpretation clusters, the continuous identification result is recorded as continuous; if the preceding and following interpretation clusters cannot be connected, causing the overall path to split, the continuous identification result is recorded as discontinuous. When the object to be removed is at the start or end point, the system compares whether the remaining interpretation clusters can form a continuous path with consistent direction, and records whether the start or end boundary has undergone irreversible changes. After completing the above comparison, the system generates a continuity identification result. The continuity identification result includes at least the current object removal identifier, the consistency identifier of interpretation direction, the existence identifier of continuous path, the number of reconstructed sub-relationships, the number of uncovered interpretation connections, the break position, boundary change information, and a judgment identifier for whether the overall interpretation relationship remains continuous. When the interpretation direction is consistent, the continuous path exists, the number of uncovered interpretation connections meets the preset requirements, and there are no irreconcilable breaks, the continuity identification result is that the overall interpretation relationship remains continuous; when there are inconsistent directions, broken paths, multiple discontinuous sub-relationships, or uncovered key interpretation connections, the continuity identification result is that the overall interpretation relationship does not remain continuous.
[0150] In a preferred embodiment of the present invention, based on the non-substitutable interpretation dataset, a verification relationship is established between each biometric data and identity information, and the identity verification result is output, including:
[0151] By matching each biometric data in the non-substitutable interpretation dataset with identity information, the identity information corresponding to each biometric data is identified, thus obtaining an identity mapping set;
[0152] Based on the identity mapping set, and according to the correspondence between the interpretation relationship of each biometric data in the non-substitutable interpretation dataset and the identity information, the verification relationship between each biometric data is established, and the verification relationship set is obtained.
[0153] Based on the verification relationship set, the consistency of identity information among various biometric data is compared, and it is identified whether each verification relationship points to the same identity information, thus obtaining the verification and identification result;
[0154] Based on the verification results, verification relationships that both point to the same identity information are identified as consistent identity, while verification relationships that do not both point to the same identity information are identified as inconsistent identity, thus obtaining the identity verification result.
[0155] In this embodiment of the invention, by matching each biometric data in the non-substitutable interpretable dataset with identity information, the identity information corresponding to each biometric data is identified, resulting in an identity mapping set. This explicitly records the identity information corresponding to each non-substitutable biometric data, providing an identity-oriented basis for subsequently establishing verification relationships between multiple biometric data. Based on the identity mapping set, and according to the correspondence between the interpretable relationships of each biometric data in the non-substitutable interpretable dataset and the identity information, verification relationships between the biometric data are established, resulting in a verification relationship set. This transforms the structural relationships between multimodal biometric data into verification relationships oriented towards identity consistency judgment. Based on the verification relationship set, each... The system identifies the consistency between identity information in biometric data and determines whether each verification relationship points to the same identity information, thus obtaining the verification result. It converts the consistency between each non-replaceable biometric data and identity information into the verification result, so that the identity verification conclusion comes from the identity pointing relationship between multiple non-replaceable biometric data. Based on the verification result, verification relationships that point to the same identity information are determined to be consistent, and verification relationships that do not point to the same identity information are determined to be inconsistent, thus obtaining the identity verification result. It has a clear data source, interpretation relationship source, and identity mapping source, enabling the identity verification process to complete identity authentication based on the screened non-replaceable interpretation data.
[0156] Specifically, by matching each biometric data in the non-substitutable interpretation dataset with identity information, the identity information corresponding to each biometric data is identified, resulting in an identity mapping set, which includes:
[0157] The system reads the data number, modality identifier, non-substitutable interpretation cluster identifier, interpretation chain number, interpretation source identifier, interpreted object identifier, and verification task identifier of each biometric data in the non-substitutable interpretation dataset. Simultaneously, it reads the identity information data in the verification dataset, which includes the person's identification number, name, identity identifier, identification photo, identification validity status, identity certificate number, and biometric reference data bound to the identity identifier. Using the verification task identifier as an index, the system binds the biometric data in the non-substitutable interpretation dataset with the identity information data under the same verification task, ensuring that subsequent matching processes are limited to the identity object corresponding to this verification.
[0158] The system determines the corresponding identity information matching method based on the modality identifier of each biometric data. For facial image data, the system extracts its facial feature representation and matches it with the ID photo or pre-stored facial template in the identity information data; for fingerprint data, the system extracts fingerprint ridge features or fingerprint encoding and matches it with the fingerprint reference template bound to the identity information; for iris data, the system extracts iris texture encoding and matches it with the iris reference template bound to the identity information; for voiceprint data, the system extracts voiceprint feature vectors and matches them with the voiceprint reference template bound to the identity information; for palm vein, palm print, gait, or other biometric data, the system calls the corresponding modality feature representation method and identity reference data for matching. For liveness detection data, the system can match it based on its corresponding facial acquisition sequence, verification object identifier, acquisition time window, and the binding relationship between the data and the facial verification process corresponding to the identity information to determine whether the liveness detection data corresponds to the current identity information.
[0159] The system treats each biometric data point in the non-substitutable interpretation dataset as a matchable data point and uses the corresponding reference data in the identity information dataset as the matching benchmark. The system identifies the identity information corresponding to the biometric data based on feature distance, similarity, encoding consistency, template matching results, or time binding relationships between the matchable data and the reference data. When the matching result meets the preset identity matching conditions, the system establishes a mapping relationship between the biometric data and the corresponding identity information identifier; when the matching result does not meet the preset identity matching conditions, or when there is no reference data for the corresponding modality in the identity information dataset, the system records the biometric data as unmapped, while retaining its non-substitutable interpretation cluster and interpretation chain information. For cases where the same biometric data matches multiple identity information points, the system can determine the corresponding identity information based on the matching result ranking, identity task binding relationship, or identity information reading source; if a unique identity information cannot be determined, it is recorded as a multi-mapping state. The system generates an identity mapping record for each biometric data point that has completed the matching process.
[0160] Specifically, based on the identity mapping set, and according to the correspondence between the interpretation relationships of each biometric data in the non-substitutable interpretation dataset and the identity information, a verification relationship set is established between the biometric data, resulting in the following:
[0161] The system reads the interpretation relationships between biometric data in the non-substitutable interpretation dataset, including the interpretation source, the interpreted object, the interpretation direction, the identifier of the non-substitutable interpretation cluster, and the interpretation chain number. Simultaneously, it reads the identity information identifier and mapping status corresponding to each biometric data in the identity mapping set. Using the interpretation relationships in the non-substitutable interpretation dataset as the structural basis and the identity information correspondence in the identity mapping set as the identity basis, the system performs cross-associations between the two. Specifically, when an interpretation relationship exists between biometric data from a certain interpretation source and biometric data of a certain interpreted object in the non-substitutable interpretation dataset, the system queries the identity information identifiers of both the interpretation source biometric data and the interpreted object biometric data in the identity mapping set. If both biometric data points are mapped to the same identity identifier, the system establishes a verification relationship indicating that the two biometric data points point to the same identity information. If they are mapped to different identity identifiers, the system establishes a verification relationship indicating that the two biometric data points point to different identities. If either data point is unmapped, the system establishes a verification relationship indicating that the identity information corresponding to this interpretation relationship is incomplete. If multiple mapping states exist, the system records multiple possible identity information points along with this interpretation relationship and marks them as verification relationships to be confirmed.
[0162] When an object to be explained is explained by multiple sources, the system establishes bidirectional verification records between each source and the object, or establishes a composite verification relationship involving multiple sources and an object. In the composite verification relationship, the system reads the identity information identifiers corresponding to all sources and the object, and records whether these biometric data correspond to the same identity information. For cases where multiple interpretation relationships exist within a non-substitutable interpretation cluster, the system establishes verification relationships sequentially according to the cluster connection order; for cases where interpretation connections exist between different non-substitutable interpretation clusters, the system establishes cross-cluster verification relationships according to the inter-cluster interpretation direction. When establishing each verification relationship, the system generates a verification relationship identifier and records the biometric data number of the source, the biometric data number of the object, the interpretation relationship direction, the interpretation cluster identifier, the interpretation chain number, the mapping status, the verification relationship type, and the verification relationship status. After establishing verification relationships for all interpretation relationships, the system aggregates all verification relationships into a verification relationship set.
[0163] Specifically, based on the verification relationship set, the consistency of identity information among various biometric data is compared, and it is identified whether each verification relationship points to the same identity information to obtain the verification and identification result, which includes:
[0164] The system reads the source biometric data ID, the interpreted object biometric data ID, the corresponding identity information identifier, the mapping status, and the verification relationship status from each verification relationship. The system sequentially traverses each verification relationship in the set and compares whether the identity information identifiers corresponding to the biometric data within the same verification relationship are the same. If both the source and the interpreted object are mapped, and their corresponding identity information identifiers are consistent, the system records the verification relationship as a consistent identity information relationship. If both the source and the interpreted object are mapped but correspond to different identity information identifiers, the system records the verification relationship as an inconsistent identity information relationship. If there are no mappings or multiple mapping states, the system records the verification relationship as an incomplete identity information relationship or a relationship awaiting confirmation. After completing the consistency comparison of a single verification relationship, the system further performs overall common-point identification on the verification relationship set. The system counts the identity information identifiers pointed to by all consistent identity information relationships and calculates the frequency of each identity information identifier in the verification relationship set, the number of biometric data involved, and the number of non-substitutable interpretation clusters covered. If all valid verification relationships in the verification relationship set point to the same identity information identifier, and there are no inconsistent identity information relationships, then the system determines that all verification relationships point to the same identity information. If there are two or more different identity information identifiers in the verification relationship set, or there are inconsistent identity information relationships, or there is biometric data in the non-substitutable interpretation dataset that cannot be mapped and affects the determination of common pointing, then the system determines that all verification relationships do not point to the same identity information.
[0165] For cases involving unconfirmed verification relationships, the system can determine whether to exclude them from common pointing identification or use them as evidence of non-common pointing based on preset rules. When the unconfirmed verification relationship involves key explanatory relationships in the non-substitutable explanatory dataset, the system records the overall verification identification result as pending review; when the unconfirmed verification relationship does not affect all confirmed verification relationships pointing to the same identity information, the system can record it as partially pending confirmation but with a valid common identity pointing. After completing the overall common pointing identification, the system generates a verification identification result. The verification identification result includes at least a verification task identifier, a common pointing identity information identifier, the number of identity information consistent relationships, the number of identity information inconsistent relationships, the number of identity information incomplete relationships, the number of unconfirmed relationships, the identifier of the non-substitutable explanatory cluster involved, the number of the biometric data involved, and a common pointing judgment identifier. The common pointing judgment identifier is used to indicate whether each verification relationship in the verification relationship set points to the same identity information.
[0166] Embodiments of the present invention also provide an identity verification system based on multimodal biometrics, the system comprising:
[0167] The data acquisition module is used to acquire personnel identity information data and multimodal biometric data to obtain a verification dataset;
[0168] The association module is used to identify the degree to which any biometric data can be jointly explained by the other biometric data, based on the verification dataset, and using any other biometric data as the target data and the remaining biometric data as the reference data, and to obtain the association response value.
[0169] The interpretation chain module is used to establish interpretation relationships by selecting biometric data whose correlation response values meet preset correlation values, connect each interpretation relationship into an interpretation chain, and determine the interpretation start point and interpretation end point to obtain an interpretation chain set;
[0170] The stabilization module is used to sequentially mask any biometric data in the interpretation chain according to the interpretation chain set, identify the ability of the biometric data to maintain the continuity of the interpretation chain, and obtain a stable contribution value.
[0171] The interpretation cluster module is used to divide biometric data with interpretation relationships and stable contribution values that meet preset succession values into the same interpretation cluster, and verify whether the biometric data within the interpretation cluster maintain the interpretation relationship, thereby obtaining an interpretation cluster set;
[0172] The replacement module is used to remove each interpretation cluster in sequence and re-establish the overall interpretation relationship based on the interpretation cluster set, and to identify whether the overall interpretation relationship remains continuous after removing a certain interpretation cluster, thus obtaining the non-replaceable interpretation dataset.
[0173] The verification module is used to establish the verification relationship between various biometric data and identity information based on the non-substitutable interpretation dataset, and output the identity verification results.
[0174] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.
[0175] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.
[0176] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.
[0177] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for identity verification based on multimodal biometrics, characterized in that, The method includes: Obtain personnel identity information data and multimodal biometric data to obtain a verification dataset; Based on the verification dataset, any biometric data is used as the target data, and the remaining biometric data are used as the reference data. The degree to which the biometric data can be jointly explained by the remaining biometric data is identified, and the associated response value is obtained. By selecting biometric data whose correlation response values meet preset correlation values, explanatory relationships are established, and each explanatory relationship is connected into an explanatory chain. The starting point and ending point of the explanatory chain are determined to obtain a set of explanatory chains. Based on the interpretation chain set, any biometric data in the interpretation chain is sequentially masked, and the ability of the biometric data to maintain the continuity of the interpretation chain is identified to obtain a stable contribution value. By dividing biometric data with explanatory relationships and stable contribution values that meet preset succession values into the same explanatory cluster, and verifying whether the biometric data within each explanatory cluster maintain explanatory relationships, an explanatory cluster set is obtained; Based on the interpretation cluster set, each interpretation cluster is removed in turn and the overall interpretation relationship is re-established. It is then identified whether the overall interpretation relationship remains continuous after a certain interpretation cluster is removed, thus obtaining the irreplaceable interpretation dataset. Based on the non-substitutable interpretation dataset, establish the verification relationship between each biometric data and identity information, and output the identity verification results.
2. The identity verification method based on multimodal biometrics according to claim 1, characterized in that, Based on the verification dataset, any biometric data is used as the target data, and the remaining biometric data are used as reference data. The degree to which the target biometric data can be jointly explained by the remaining biometric data is identified, and the associated response value is obtained, including: By calculating the data retention ratio of each target data unit, the degree to which the target data is retained under the combined effect of reference data is identified, and the retention ratio is obtained; by calculating the source coverage ratio of each target data unit, the coverage capability of the target data to be jointly interpreted by multiple reference data is identified, and the source coverage is obtained. By calculating the continuous retention ratio of each target data unit, the stability of the target data in the common interpretation process is identified, and the continuous retention term is obtained; by calculating the deviation of the retention ability among each reference data, the distribution balance of the explanatory power of each reference data on the target data is identified, and the source balance term is obtained. By integrating the percentage, source coverage, continuity, and source balance terms, the degree to which the target data is jointly explained by other biometric data is identified, and the associated response value is obtained.
3. The identity verification method based on multimodal biometrics according to claim 2, characterized in that, By selecting biometric data whose correlation response values satisfy a preset correlation value, explanatory relationships are established. These explanatory relationships are then connected to form explanatory chains, and the starting and ending points of the explanatory chains are determined, resulting in a set of explanatory chains, including: By comparing the correlation response value of each biometric data with the preset correlation value, the biometric data that are greater than the preset correlation value are identified as candidate explanatory data, thus obtaining the candidate explanatory dataset; Based on the candidate explanation dataset, explanation relationships are established according to the common explanation objects among the candidate explanation data, and the explanation source and the explained object of each explanation relationship are recorded to obtain the explanation relationship set; Based on the set of explanatory relations, the explanatory relations that correspond continuously to the source of explanation and the object being explained are connected sequentially, while keeping the direction of explanation of each explanatory relation consistent, to obtain the initial explanatory chain; Based on the initial interpretation chain, identify the interpretation sources that are not the objects being interpreted as the starting point of the interpretation, and identify the objects being interpreted that are not the sources of the interpretation as the ending point of the interpretation, thus obtaining the interpretation chain set.
4. The identity verification method based on multimodal biometrics according to claim 3, characterized in that, Based on the interpretation chain set, any biometric data in the interpretation chain is sequentially masked to identify the ability of that biometric data to maintain the continuity of the interpretation chain, thus obtaining a stable contribution value, including: Based on the set of interpretation chains, any biometric data in the interpretation chain is sequentially masked, and the number of interpretation relationships that are broken due to masking the biometric data in each interpretation chain is counted. The support capability of the biometric data for the connection structure of the interpretation chain is identified, and the link breakage item is obtained. The ability of the original connection relationship of the interpretation chain to be maintained after masking the biometric data is identified, and the link maintenance item is obtained. Based on the number of changes in the starting and ending points of each interpretation chain after the biometric data is masked, the degree of change in the boundary structure of the interpretation chain after masking the biometric data is identified, and the endpoint change term is obtained; the degree of adjustment of the internal structure of the interpretation chain after masking the biometric data is identified, and the position perturbation term is obtained. By fusing link breakage, link maintenance, endpoint change, and location disturbance items, and identifying the degree of disturbance in the overall chain structure after masking the biometric data, a stable contribution value is obtained.
5. The identity verification method based on multimodal biometrics according to claim 4, characterized in that, By dividing biometric data with explanatory relationships and stable contribution values that meet preset succession values into the same explanatory cluster, and verifying whether the biometric data within each explanatory cluster maintain explanatory relationships, an explanatory cluster set is obtained, including: By comparing the stable contribution value of each biometric data with the preset succession value, biometric data that can maintain a continuous interpretive relationship are identified, and a succession dataset is obtained. Based on the continuation dataset, biometric data with continuous interpretation relationships are grouped according to the source of interpretation, the object being interpreted, and the order of interpretation connections among the biometric data. Biometric data in the same continuous interpretation relationship are identified to obtain an initial interpretation cluster. Based on the initial interpretation clusters, each biometric data in each initial interpretation cluster is sequentially masked, and the original interpretation relationship between the remaining biometric data is identified to obtain the intra-cluster verification results. Based on the intra-cluster validation results, biometric data that retain the original interpretation relationship are retained in the initial interpretation cluster, while biometric data that do not retain the original interpretation relationship are removed from the initial interpretation cluster, resulting in an interpretation cluster set.
6. The identity verification method based on multimodal biometrics according to claim 5, characterized in that, Based on the set of explanation clusters, each explanation cluster is removed sequentially, and the overall explanation relationship is re-established. It is then determined whether the overall explanation relationship remains continuous after removing a particular explanation cluster, resulting in a dataset of irreplaceable explanations, including: By sequentially selecting any one of the interpretation clusters in the interpretation cluster set as the removal object and removing the removal object from the interpretation cluster set, the interpretation connection relationship between the remaining interpretation clusters is identified, and the removal interpretation cluster set is obtained. The overall interpretation relationship is re-established by recognizing the interpretation sources, interpreted objects, and interpretation connection order among the removed interpretation clusters, and it is then identified whether this overall interpretation relationship covers the interpretation connections before removal, thus obtaining the reconstructed interpretation relationship. Based on the reconstructed interpretation relationship, the continuity between the overall interpretation relationship and the interpretation connection before removal is compared, and it is identified whether the overall interpretation relationship remains continuous after the object is removed, thus obtaining the continuity identification result. Based on the continuous identification results, the objects whose overall interpretation relationship remains continuous are identified as alternative interpretation clusters, and the objects whose overall interpretation relationship does not remain continuous are identified as non-alternative interpretation clusters. The biometric data in each non-alternative interpretation cluster are collected to obtain the non-alternative interpretation dataset.
7. The identity verification method based on multimodal biometrics according to claim 6, characterized in that, Based on the non-substitutable interpretation dataset, establish the verification relationship between each biometric data and identity information, and output the identity verification results, including: By matching each biometric data in the non-substitutable interpretation dataset with identity information, the identity information corresponding to each biometric data is identified, thus obtaining an identity mapping set; Based on the identity mapping set, and according to the correspondence between the interpretation relationship of each biometric data in the non-substitutable interpretation dataset and the identity information, the verification relationship between each biometric data is established, and the verification relationship set is obtained. Based on the verification relationship set, the consistency of identity information among various biometric data is compared, and it is identified whether each verification relationship points to the same identity information, thus obtaining the verification and identification result; Based on the verification results, verification relationships that both point to the same identity information are identified as consistent identity, while verification relationships that do not both point to the same identity information are identified as inconsistent identity, thus obtaining the identity verification result.
8. An identity verification system based on multimodal biometrics, characterized in that, The system is used to perform the method as described in any one of claims 1 to 7, the system comprising: The data acquisition module is used to acquire personnel identity information data and multimodal biometric data to obtain a verification dataset; The association module is used to identify the degree to which any biometric data can be jointly explained by the other biometric data, based on the verification dataset, and using any other biometric data as the target data and the remaining biometric data as the reference data, and to obtain the association response value. The interpretation chain module is used to establish interpretation relationships by selecting biometric data whose correlation response values meet preset correlation values, connect each interpretation relationship into an interpretation chain, and determine the interpretation start point and interpretation end point to obtain an interpretation chain set; The stabilization module is used to sequentially mask any biometric data in the interpretation chain according to the interpretation chain set, identify the ability of the biometric data to maintain the continuity of the interpretation chain, and obtain a stable contribution value. The interpretation cluster module is used to divide biometric data with interpretation relationships and stable contribution values that meet preset succession values into the same interpretation cluster, and verify whether the biometric data within the interpretation cluster maintain the interpretation relationship, thereby obtaining an interpretation cluster set; The replacement module is used to remove each interpretation cluster in sequence and re-establish the overall interpretation relationship based on the interpretation cluster set, and to identify whether the overall interpretation relationship remains continuous after removing a certain interpretation cluster, thus obtaining the non-replaceable interpretation dataset. The verification module is used to establish the verification relationship between various biometric data and identity information based on the non-substitutable interpretation dataset, and output the identity verification results.
9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.
