Multi-mode fusion authentication and recognition system of intelligent ticket checking gate

Abstract

The invention discloses a multi-modal fusion authentication and recognition system of an intelligent ticket gate, and relates to the technical field of multi-modal fusion authentication and recognition, the system comprises a data preprocessing module, a variance verification module, a coefficient verification module and an error analysis module, the data preprocessing module receives a face image, identity card information and a two-dimensional code scanning result, and sends the face image, the identity card information and the two-dimensional code scanning result to the variance verification module; the method comprises the following steps of: cleaning data, eliminating noise, defining face image information A, identity card information B and a two-dimensional code scanning result C, and performing variance verification and coefficient verification on the face image, the identity card information and the two-dimensional code scanning result, so that the error of single verification is avoided, the accuracy of signals is ensured, and meanwhile, key information is prevented from being lost through comprehensive error analysis; in addition, repeated calculation of redundant information is effectively avoided, authentication and recognition are further verified comprehensively, and the verification effect of the multi-mode fusion authentication and recognition system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal fusion authentication and recognition, and in particular to a multimodal fusion authentication and recognition system for an intelligent ticket gate. Background Art

[0002] The multimodal fusion authentication and recognition system of the intelligent ticket gate provides accurate and fast recognition services for passengers or tourists by integrating information from multiple sensors and modalities such as images, sounds, and touch. This system uses advanced multimodal fusion technology, which can integrate and fuse information from different sensors to generate a more comprehensive, accurate, and reliable data set.

[0003] In the actual application of intelligent ticket gates, multimodal fusion technology covers a variety of recognition methods, including image recognition (such as face recognition), ID card verification (such as reading ID card information), and QR code scanning. These methods work together to form a multi-dimensional recognition system for passengers or tourists, improving ticket checking efficiency and accuracy.

[0004] However, in actual operations, data from different modalities often contain redundant information or lose key information. The existence of redundant information will increase the computational burden and reduce system efficiency; while the loss of key information will lead to a decline in recognition performance and affect the accuracy of ticket checking results.

[0005] In addition, there is a temporal misalignment in data of different modalities, such as the asynchrony between video and audio data. This asynchrony can cause difficulties in fusion, seriously affecting the effectiveness of the multimodal fusion authentication and recognition system of the intelligent ticket gate. Summary of the invention

[0006] In view of the above situation, the present invention ensures the accuracy of the signal through variance check, coefficient verification and multiple verification methods for facial images, ID card information and QR code scanning results. At the same time, it prevents the loss of key information through comprehensive error analysis and effectively avoids the repeated calculation of redundant information.

[0007] The technical solution includes a data preprocessing module, a variance check module, a coefficient verification module and an error analysis module. The data preprocessing module receives a face image, ID card information and a QR code scanning result, cleans the data, eliminates noise, and defines face image information A, ID card information B and QR code scanning result C. Extract the corresponding face image threshold information A1, ID card threshold information B1, and QR code scanning threshold result C1 from the database, and calculate the face image error ratio A2, A2=(A1-A) / A1, ID card information error ratio B2, B2=(B1-B) / B1, and QR code scanning result error ratio C2, C2=(C1-C) / C1; The variance check module calculates the average X of A2, B2, and C2, X=(A2+B2+C2) / 3, and then calculates the variance S of A2, B2, and C2. ,Extract the corresponding variance threshold M from the database. If S is greater than M, it means that the data difference in this group is large. Calculate the variance error ratio S1, S1= (SM) / M. On the contrary, if S is less than or equal to M, it means that the data difference in this group is normal; The coefficient verification module calculates the coefficient error value W, W=A2×0.4+B2×0.3+C2×0.3, extracts the corresponding coefficient error threshold W1 from the database, W greater than W1 indicates that the coefficient verification of this group of data is abnormal, calculates the coefficient error ratio W1, W1=(W-W1) / W1, conversely, W is less than or equal to W1, indicating that the coefficient verification of this group of data is normal; When the error analysis module receives the normal results of the difference and coefficient verification of this group of data, it indicates that the overall verification of this group of data is normal; otherwise, the comprehensive error value H is calculated, H=0.6×S1+0.4×W1, H greater than 0.3 indicates that the error of this group of data is large, and a warning signal is fed back to the corresponding ticket gate, and manual re-authentication and identification are performed. After manual verification, the person is allowed to enter the park; H is less than or equal to 0.3, indicating that the error of this group of data is within the controllable range, and a maintenance signal is fed back to the corresponding ticket gate to notify relevant personnel to repair the ticket gate.

[0008] Furthermore, it also includes a visualization control module, which displays the verification results and the personnel information entering the park in a visualized form through the verified face image, ID card information, and QR code scanning results. At the same time, the personnel information that has been manually re-authenticated and passed is also displayed in a visualized form; When notifying relevant personnel to inspect and repair the ticket gate, the module will display a prompt that the corresponding ticket gate is "under maintenance", and at the same time notify the ticket entrance to suspend the queue of people at this ticket gate to avoid congestion and chaos.

[0009] Furthermore, the data preprocessing module receives the face image, ID card information, and QR code scanning results, cleans the data, eliminates noise, and sorts them in sequence according to the response time of the ticket gate, collects a group of corresponding face images, ID card information, and QR code scanning results in response, and checks the authentication and identification information of each group in sequence according to the sorted response time. When invalid or blank identification information is screened out, an early warning signal is sent to the corresponding ticket gate, and the identification information is re-authenticated until the corresponding face image, ID card information, and QR code scanning results can correctly identify the information; The facial image, ID card information, and QR code scanning result of each group are encoded and then converted into numerical information to unify the dimensions of the authentication and identification information, which are defined as facial image information A, ID card information B, and QR code scanning result C respectively.

[0010] Furthermore, the data are sorted in sequence according to the response time of ticket verification at the ticket gate. Every time the ticket gate responds to a ticket verification request, the system will collect and integrate the corresponding facial image, ID card information and QR code scanning results to form a complete set of data. In this set of data, the response time of the facial image, ID card information and QR code scanning results are all uniformly recognized as the earliest response time in the set of data.

[0011] Furthermore, when any one of the facial image error ratio A2, the ID card information error ratio B2 or the QR code scanning result error ratio C2 shows a negative value, this indicates that there is an abnormality in the received facial image, ID card information or QR code scanning result. The further verification process is immediately terminated, the corresponding ticket gate is located, and the relevant personnel are notified to inspect the ticket gate. At the same time, the ticket entrance personnel are notified to suspend the queue of people at this ticket gate.

[0012] Furthermore, if the manual re-authentication is still abnormal, the person will be prohibited from entering the park and the authentication and identification information of the person will be deleted from the database; Manually re-authenticate and identify the personnel information that has passed the verification, mark the personnel information as a negative sample, and use the negative sample to correct the corresponding threshold model in the system. By continuously adjusting and optimizing the parameters of the model, fine-tune the threshold model and dynamically adjust the threshold in the system. This system directly authenticates and identifies the verified personnel information, marks this personnel information as a positive sample, optimizes and upgrades the corresponding threshold model through positive samples, and dynamically adjusts the threshold in this system.

[0013] Due to the adoption of the above technical solution, the present invention has the following advantages compared with the prior art: 1. Sort by the response time of ticket inspection at the ticket gate. Every time the ticket gate responds to a ticket inspection request, the system will collect and integrate the corresponding facial image, ID card information and QR code scanning results to form a complete set of data. In this set of data, the response time of the facial image, ID card information and QR code scanning results are uniformly identified as the earliest response time in the set of data, effectively preventing the misalignment of data in different modes in time, and improving the use effect of the multi-modal fusion authentication and recognition system of the intelligent ticket gate; 2. The face image, ID card information, and QR code scanning results are verified through variance check and coefficient verification to avoid the error of single verification and ensure the accuracy of the signal. At the same time, comprehensive error analysis is used to prevent the loss of key information and effectively avoid repeated calculation of redundant information. Further comprehensive verification of authentication and identification is performed to improve the verification effect of the multi-modal fusion authentication and identification system; 3. The personnel information that has been manually re-authenticated and identified is marked as a negative sample, and the corresponding threshold model in this system is corrected with the negative sample. By continuously adjusting and optimizing the parameters of the model, the threshold model is fine-tuned, and the threshold in this system is dynamically adjusted, thereby improving the practical value of the multimodal fusion authentication and identification system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The present invention is a flow chart of a multi-modal fusion authentication and recognition system for an intelligent ticket gate. DETAILED DESCRIPTION

[0015] The above and other technical contents, features and effects of the present invention are described in detail below with reference to the attached Figure 1 The detailed description of the embodiments will clearly show that the structural contents mentioned in the following embodiments are all based on the drawings in the specification.

[0016] In order to prevent the existence of redundant information from increasing the computational burden and reducing system efficiency, and to prevent the data of different modalities from being misaligned in time, a data preprocessing module, a variance check module, a coefficient verification module and an error analysis module are included. The data preprocessing module receives the face image, the ID card information and the QR code scanning result, cleans the data, eliminates noise, and defines the face image information A, the ID card information B and the QR code scanning result C; Extract the corresponding face image threshold information A1, ID card threshold information B1, and QR code scanning threshold result C1 from the database, and calculate the face image error ratio A2, A2=(A1-A) / A1, ID card information error ratio B2, B2=(B1-B) / B1, and QR code scanning result error ratio C2, C2=(C1-C) / C1; The variance check module calculates the average X of A2, B2, and C2, X=(A2+B2+C2) / 3, and then calculates the variance S of A2, B2, and C2. ,Extract the corresponding variance threshold M from the database. If S is greater than M, it means that the data difference in this group is large. Calculate the variance error ratio S1, S1= (SM) / M. On the contrary, if S is less than or equal to M, it means that the data difference in this group is normal; The coefficient verification module calculates the coefficient error value W, W=A2×0.4+B2×0.3+C2×0.3, extracts the corresponding coefficient error threshold W1 from the database, W greater than W1 indicates that the coefficient verification of this group of data is abnormal, calculates the coefficient error ratio W1, W1=(W-W1) / W1, conversely, W is less than or equal to W1, indicating that the coefficient verification of this group of data is normal; When the error analysis module receives the normal results of the difference and coefficient verification of this group of data, it indicates that the overall verification of this group of data is normal; otherwise, the comprehensive error value H is calculated, H=0.6×S1+0.4×W1, H greater than 0.3 indicates that the error of this group of data is large, and a warning signal is fed back to the corresponding ticket gate, and manual re-authentication and identification are performed. After manual verification, the person is allowed to enter the park; H is less than or equal to 0.3, indicating that the error of this group of data is within the controllable range, and a maintenance signal is fed back to the corresponding ticket gate to notify relevant personnel to repair the ticket gate.

[0017] Furthermore, it also includes a visualization control module, which displays the verification results and the personnel information entering the park in a visualized form through the verified face image, ID card information, and QR code scanning results. At the same time, the personnel information that has been manually re-authenticated and passed is also displayed in a visualized form; When notifying relevant personnel to inspect and repair the ticket gate, the module will display a prompt that the corresponding ticket gate is "under maintenance", and at the same time notify the ticket entrance to suspend the queue of people at this ticket gate to avoid congestion and chaos.

[0018] Furthermore, the data preprocessing module receives the face image, ID card information, and QR code scanning results, cleans the data, eliminates noise, and sorts them in sequence according to the response time of the ticket gate, collects a group of corresponding face images, ID card information, and QR code scanning results in response, and checks the authentication and identification information of each group in sequence according to the sorted response time. When invalid or blank identification information is screened out, an early warning signal is sent to the corresponding ticket gate, and the identification information is re-authenticated until the corresponding face image, ID card information, and QR code scanning results can correctly identify the information; The facial image, ID card information, and QR code scanning result of each group are encoded and then converted into numerical information to unify the dimensions of the authentication and identification information, which are defined as facial image information A, ID card information B, and QR code scanning result C respectively.

[0019] Furthermore, the data are sorted in sequence according to the response time of ticket verification at the ticket gate. Every time the ticket gate responds to a ticket verification request, the system will collect and integrate the corresponding facial image, ID card information and QR code scanning results to form a complete set of data. In this set of data, the response time of the facial image, ID card information and QR code scanning results are all uniformly recognized as the earliest response time in the set of data.

[0020] Furthermore, when any one of the facial image error ratio A2, the ID card information error ratio B2 or the QR code scanning result error ratio C2 shows a negative value, this indicates that there is an abnormality in the received facial image, ID card information or QR code scanning result. The further verification process is immediately terminated, the corresponding ticket gate is located, and the relevant personnel are notified to inspect the ticket gate. At the same time, the ticket entrance personnel are notified to suspend the queue of people at this ticket gate.

[0021] Furthermore, if the manual re-authentication is still abnormal, the person will be prohibited from entering the park and the authentication and identification information of the person will be deleted from the database; Manually re-authenticate and identify the personnel information that has passed the verification, mark the personnel information as a negative sample, and use the negative sample to correct the corresponding threshold model in the system. By continuously adjusting and optimizing the parameters of the model, fine-tune the threshold model and dynamically adjust the threshold in the system. This system directly authenticates and identifies the verified personnel information, marks this personnel information as a positive sample, optimizes and upgrades the corresponding threshold model through positive samples, and dynamically adjusts the threshold in this system.

[0022] When the present invention is used specifically, based on the existing technology, the data preprocessing module receives the face image, ID card information, and QR code scanning result, cleans the data, eliminates noise, and defines the face image information A, ID card information B, and QR code scanning result C; Extract the corresponding face image threshold information A1, ID card threshold information B1, and QR code scanning threshold result C1 from the database, and calculate the face image error ratio A2, A2=(A1-A) / A1, ID card information error ratio B2, B2=(B1-B) / B1, and QR code scanning result error ratio C2, C2=(C1-C) / C1; The variance check module calculates the average X of A2, B2, and C2, X=(A2+B2+C2) / 3, and then calculates the variance S of A2, B2, and C2. ,Extract the corresponding variance threshold M from the database. If S is greater than M, it means that the data difference in this group is large. Calculate the variance error ratio S1, S1= (SM) / M. On the contrary, if S is less than or equal to M, it means that the data difference in this group is normal; The coefficient verification module calculates the coefficient error value W, W=A2×0.4+B2×0.3+C2×0.3, extracts the corresponding coefficient error threshold W1 from the database, W greater than W1 indicates that the coefficient verification of this group of data is abnormal, calculates the coefficient error ratio W1, W1=(W-W1) / W1, conversely, W is less than or equal to W1, indicating that the coefficient verification of this group of data is normal; When the error analysis module receives the normal results of the difference and coefficient verification of this group of data, it indicates that the overall verification of this group of data is normal; otherwise, the comprehensive error value H is calculated, H=0.6×S1+0.4×W1, H greater than 0.3 indicates that the error of this group of data is large, and a warning signal is fed back to the corresponding ticket gate, and manual re-authentication and identification are performed. After manual verification, the person is allowed to enter the park; H is less than or equal to 0.3, indicating that the error of this group of data is within the controllable range, and a maintenance signal is fed back to the corresponding ticket gate to notify relevant personnel to repair the ticket gate.

[0023] It also includes a visualization control module, which displays the verification results and the personnel information entering the park in a visualized form using the verified face image, ID card information, and QR code scanning results. At the same time, the personnel information that has been manually re-authenticated and passed is also displayed in a visualized form; When notifying relevant personnel to inspect the ticket gate, the module will display the prompt "under maintenance" for the corresponding ticket gate, and notify the ticket entrance to suspend the queue of personnel at this ticket gate to avoid congestion and chaos. Using the above method, the face image, ID card information, and QR code scanning results are verified by variance verification, coefficient verification, and multiple verification methods to ensure the accuracy of the signal. At the same time, comprehensive error analysis is used to prevent the loss of key information and effectively avoid the repeated calculation of redundant information.

[0024] The above is a further detailed description of the present invention in combination with a specific implementation method, and it cannot be determined that the specific implementation of the present invention is limited to this; for technical personnel in the technical field to which the present invention belongs and related technical fields, based on the technical solution of the present invention, the expansion and replacement of operating methods and data should all fall within the protection scope of the present invention.

Claims

1. A multi-modal fusion authentication and recognition system for an intelligent ticket gate, characterized in that: It includes a data preprocessing module, a variance check module, a coefficient verification module, an error analysis module, and a visualization control module. The data preprocessing module receives a face image, ID card information, and a QR code scanning result, cleans the data, eliminates noise, and defines face image information A, ID card information B, and QR code scanning result C; Extract the corresponding face image threshold information A1, ID card threshold information B1, and QR code scanning threshold result C1 from the database, and calculate the face image error ratio A2, A2=(A1-A) / A1, ID card information error ratio B2, B2=(B1-B) / B1, and QR code scanning result error ratio C2, C2=(C1-C) / C1; The variance check module calculates the average X of A2, B2, and C2, X=(A2+B2+C2) / 3, and then calculates the variance S of A2, B2, and C2. ,Extract the corresponding variance threshold M from the database. If S is greater than M, it means that the data difference in this group is large. Calculate the variance error ratio S1, S1= (SM) / M. On the contrary, if S is less than or equal to M, it means that the data difference in this group is normal; The coefficient verification module calculates the coefficient error value W, W=A2×0.4+B2×0.3+C2×0.3, extracts the corresponding coefficient error threshold W1 from the database, W greater than W1 indicates that the coefficient verification of this group of data is abnormal, calculates the coefficient error ratio W1, W1=(W-W1) / W1, conversely, W is less than or equal to W1, indicating that the coefficient verification of this group of data is normal; When the error analysis module receives the normal difference and coefficient verification results of the current group of data, it indicates that the overall verification of the current group of data is normal, and sends the verification results to the visualization control module; The visualization control module displays the verification results and the personnel information entering the park in a visualized form using the verified face image, ID card information, and QR code scanning results. At the same time, the personnel information that has been manually re-authenticated and recognized is also displayed in a visualized form; When notifying relevant personnel to inspect and repair the ticket gate, the module will display a prompt that the corresponding ticket gate is "under maintenance", and at the same time notify the ticket entrance to suspend the queue of people at this ticket gate to avoid congestion and chaos.

2. According to claim 1, a multi-modal fusion authentication and recognition system for an intelligent ticket gate is characterized in that: When the error analysis module receives a large difference in the data group or an abnormal coefficient verification result, it calculates the comprehensive error value H, H=0.6×S1+0.4×W1. If H is greater than 0.3, it means that the error of the data group is large. A warning signal is fed back to the corresponding ticket gate, and manual re-authentication and identification are performed. After manual verification, the person is allowed to enter the park. If H is less than or equal to 0.3, it means that the error of the data group is within the controllable range. A maintenance signal is fed back to the corresponding ticket gate to notify relevant personnel to repair the ticket gate.

3. According to the multi-modal fusion authentication and recognition system of the intelligent ticket gate according to claim 1, it is characterized in that: The data preprocessing module receives the face image, ID card information, and QR code scanning results, cleans the data, eliminates noise, and sorts them in sequence according to the response time of the ticket gate, collects a group of corresponding face images, ID card information, and QR code scanning results in response, and checks the authentication and identification information of each group in sequence according to the sorted response time. When invalid or blank identification information is screened out, a warning signal is sent to the corresponding ticket gate to re-authenticate the identification information until the corresponding face image, ID card information, and QR code scanning results can correctly identify the information; The facial image, ID card information, and QR code scanning result of each group are encoded and then converted into numerical information to unify the dimensions of the authentication and identification information, which are defined as facial image information A, ID card information B, and QR code scanning result C respectively.

4. According to claim 3, a multi-modal fusion authentication and recognition system for an intelligent ticket gate is characterized in that: The data are sorted in sequence according to the response time of ticket verification at the ticket gate. Every time the ticket gate responds to a ticket verification request, the system will collect and integrate the corresponding facial image, ID card information and QR code scanning results to form a complete set of data. In this set of data, the response time of the facial image, ID card information and QR code scanning results are uniformly recognized as the earliest response time in the set of data.

5. According to claim 1, a multi-modal fusion authentication and recognition system for an intelligent ticket gate is characterized in that: When any of the facial image error ratio A2, the ID card information error ratio B2 or the QR code scanning result error ratio C2 shows a negative value, this indicates that there is an abnormality in the received facial image, ID card information or QR code scanning result. The further verification process is immediately terminated, the corresponding ticket gate is located, and the relevant personnel are notified to inspect the ticket gate. At the same time, the ticket entrance personnel are notified to suspend the queue of people at this ticket gate.

6. According to claim 1, a multi-modal fusion authentication and recognition system for an intelligent ticket gate is characterized in that: If the manual re-authentication is still abnormal, the person will be prohibited from entering the park and the authentication information of the person will be deleted from the database; Manually re-authenticate and identify the personnel information that has passed the verification, mark the personnel information as a negative sample, and use the negative sample to correct the corresponding threshold model in the system. By continuously adjusting and optimizing the parameters of the model, fine-tune the threshold model and dynamically adjust the threshold in the system. This system directly authenticates and identifies the verified personnel information, marks this personnel information as a positive sample, optimizes and upgrades the corresponding threshold model through positive samples, and dynamically adjusts the threshold in this system.

Images