Railway diversified payment method and system based on multimodal recognition

Abstract

In the field of multimodal recognition technology, the present application provides a railway diversified payment method and system based on multimodal recognition to solve the problems of low payment verification efficiency and poor traffic security of existing railway gates during peak hours. The method of the present invention includes: synchronously collecting dynamic QR code data, real-time user voice data, facial image data and IC card data at the gate in the railway station; converting the dynamic QR code data into a dynamic token coding sequence with a timestamp, extracting multi-dimensional biometric feature vectors from the voice and face data, and parsing the identity verification code from the IC card; multimodally fusing the dynamic token, identity verification code and biometric feature vector to generate a composite verification identifier, and dynamically adjusting the verification delay threshold according to the real-time passenger flow; when the composite verification identifier matches the pre-stored authorization information and the delay threshold adapts to the current passenger flow safety interval, the gate is triggered to open. The present application improves the payment verification efficiency and traffic security of railway gates during peak hours.

Description

Technical Field

[0001] The present application relates to the field of multimodal recognition technology, and in particular to a railway diversified payment method and system based on multimodal recognition. Background Art

[0002] As railway passenger volume continues to grow, in-station payment gates must simultaneously meet the demands of high security, high efficiency, and compatibility with multiple payment methods. Passengers must be protected from payment risks such as QR code counterfeiting and IC card theft, while also supporting contactless authentication methods like voiceprint and facial recognition. Verification response speeds must also be dynamically adjusted during peak traffic hours to avoid congestion.

[0003] Existing solutions utilize a dual verification mechanism using static QR codes and facial recognition. This mechanism scans the QR code on a passenger's phone to obtain payment information and simultaneously captures a facial image for identity verification. The system matches the QR code data with pre-stored facial features. Once both are verified, the gate is released. A fixed time threshold is set to provide a uniform verification processing time for all passengers.

[0004] This solution's static QR codes are susceptible to copying and replay attacks, and facial recognition performance degrades when passengers wear masks or in low light. Fixed verification times cannot adapt to real-time fluctuations in passenger flow, leading to queues and backlogs during peak hours and idle system resources during off-peak hours. Dual verification simply involves parallel processing, resulting in insufficient overall anti-counterfeiting strength. Summary of the Invention

[0005] The present application provides a railway diversified payment method and system based on multimodal recognition, which is used to solve the problems of low payment verification efficiency and poor traffic security of railway gates during peak hours in the existing technology.

[0006] In a first aspect, the present application provides a railway diversified payment method based on multimodal recognition, comprising:

[0007] Collect dynamic QR code data, user real-time voice data, user face image data, and IC card data at the gate in the railway station;

[0008] Generate a dynamic token encoding sequence based on the dynamic QR code data, and generate a multi-dimensional biometric feature vector based on the user's real-time voice data and the user's facial image data;

[0009] Extracting the identity verification code from the IC card data, performing multimodal fusion on the dynamic token encoding sequence, the identity verification code and the multi-dimensional biometric feature vector to generate a composite verification identifier;

[0010] Adjusting the delay threshold between the composite verification mark and the gate's response action based on the dynamic change in passenger traffic flow within the railway station;

[0011] When the matching result between the composite verification identifier and the pre-stored authorization information meets the preset composite constraint conditions, and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger traffic flow, the payment verification success status of the gate is activated and the gate is controlled to open.

[0012] Optionally, extracting the identity verification code from the IC card data, performing multimodal fusion on the dynamic token encoding sequence, the identity verification code, and the multi-dimensional biometric feature vector to generate a composite verification identifier includes:

[0013] Performing encryption protocol parsing on the IC card data to extract an encrypted string, and converting the encrypted string into an identity verification code of a first fixed length;

[0014] The dynamic token encoding sequence, the identity verification code and the multi-dimensional biometric feature vector are arranged in layers and combined with a preset secure communication protocol to generate a composite verification mark.

[0015] Optionally, the step of arranging the dynamic token encoding sequence, the identity verification code, and the multi-dimensional biometric feature vector in layers and combining them with a preset secure communication protocol to generate a composite verification identifier includes:

[0016] Converting the dynamic token encoding sequence into a header identifier of a second fixed length;

[0017] Converting the identity verification code into an intermediate verification segment;

[0018] Forming a tail verification segment of the multi-dimensional biometric feature vector according to a preset dimensional order;

[0019] Based on a preset secure communication protocol, the header identifier, the middle verification segment, and the tail verification segment are sequentially spliced ​​and redundantly verified to generate a composite verification identifier.

[0020] Optionally, the sequential concatenation and redundancy checking of the header identifier, the middle verification segment, and the tail verification segment based on a preset secure communication protocol to generate a composite verification identifier includes:

[0021] According to the field sequence rule defined by the secure communication protocol, the header identifier, the intermediate check segment, and the tail verification segment are spliced ​​in a physical order from head to tail to generate an initial data block;

[0022] Performing a joint check on a first parity check bit of a header identifier, a second parity check bit of a middle verification segment, and a third parity check bit of a tail verification segment in the initial data block;

[0023] When the first parity check bit, the second parity check bit, and the third parity check bit all pass verification, appending a global redundancy check code to the end of the initial data block, where the global redundancy check code is generated by performing a modular operation based on the byte length of the header identifier, the hash value of the middle check segment, and the number of characteristic dimensions of the tail verification segment;

[0024] The data block after the global redundancy check code is added is marked as a composite verification identifier.

[0025] Optionally, generating a dynamic token encoding sequence based on the dynamic QR code data, and generating a multi-dimensional biometric feature vector based on the user's real-time voice data and the user's facial image data, includes:

[0026] Binding the time-sensitive parameters in the dynamic QR code data with the device identifier of the gate in the railway station to generate a dynamic token code sequence;

[0027] Performing waveform segmentation processing on the real-time voice data of the user, extracting a first characteristic component from the waveform segmentation processing result, and performing contour reference point calibration on the facial image data of the user, and extracting a second characteristic component from the contour reference point calibration result;

[0028] The first feature component and the second feature component are superimposed according to a preset weight ratio to generate a multi-dimensional biometric feature vector.

[0029] Optionally, adjusting the delay threshold between the composite verification identifier and the gate machine response action based on the dynamic change of the passenger flow in the railway station includes:

[0030] Real-time monitoring of passenger density parameters in the gate area of ​​a railway station, determining a dynamic change in passenger flow based on the passenger density parameters, and calculating a safe passage interval between passengers based on the dynamic change;

[0031] Based on the relationship between the hierarchical structure complexity of the composite verification identifier and the inverse of the safe passage interval, the delay threshold is dynamically adjusted. If the adjustment direction is to reduce the delay threshold, it means shortening the verification time window of the head identifier and the tail verification segment in the composite verification identifier so that the delay threshold is negatively correlated with the safe passage interval.

[0032] Optionally, when the matching result between the composite verification identifier and the pre-stored authorization information satisfies a preset composite constraint condition, and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger passage flow, activating the payment verification success state of the gate and controlling the gate to open includes:

[0033] Match the header identifier, the middle check segment and the tail verification segment in the composite verification identifier with the dynamic token whitelist, the IC card identity library and the biometric template in the pre-stored authorization information segment by segment;

[0034] When the header identifier exists in the dynamic token whitelist, the intermediate verification segment matches any record in the IC card identity database, and the similarity between the tail verification segment and the biometric template exceeds a preset ratio, it is determined that the composite constraint condition is satisfied;

[0035] Synchronously detect whether the current value of the adjusted delay threshold is within the allowable range corresponding to the safe passage interval. If it is satisfied, send an opening instruction to the gate control unit and mark the current payment verification status as successfully activated.

[0036] In a second aspect, the present application provides a railway diversified payment system based on multimodal recognition, comprising:

[0037] The acquisition module is used to collect dynamic QR code data, user real-time voice data, user face image data and IC card data at the gate in the railway station;

[0038] A generation module, configured to generate a dynamic token encoding sequence based on the dynamic QR code data, and generate a multi-dimensional biometric feature vector based on the user's real-time voice data and the user's facial image data;

[0039] An extraction module, configured to extract the identity verification code from the IC card data, and perform multimodal fusion of the dynamic token encoding sequence, the identity verification code, and the multi-dimensional biometric feature vector to generate a composite verification identifier;

[0040] An adjustment module, configured to adjust a delay threshold between the composite verification identifier and a gate response action based on a dynamic change in passenger flow within the railway station;

[0041] The activation module is used to activate the payment verification success status of the gate and control the gate to open when the matching result of the composite verification identifier and the pre-stored authorization information meets the preset composite constraint conditions and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger traffic flow.

[0042] In a third aspect, the present application provides a computing device comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a railway diversified payment method based on multimodal recognition as described in any one of the first aspects.

[0043] In a fourth aspect, the present application provides a computer storage medium having computer program instructions stored thereon, which, when executed by a processor, implements a railway diversified payment method based on multimodal recognition as described in any one of the first aspects.

[0044] In the present application, a railway diversified payment method based on multimodal recognition is provided, which includes: collecting dynamic QR code data, user real-time voice data, user face image data and IC card data at a gate in a railway station; generating a dynamic token coding sequence based on the dynamic QR code data, and generating a multi-dimensional biometric feature vector based on the user real-time voice data and the user face image data; extracting an identity verification code from the IC card data, multimodally fusing the dynamic token coding sequence, the identity verification code and the multi-dimensional biometric feature vector to generate a composite verification identifier; adjusting the delay threshold between the composite verification identifier and the gate response action based on the dynamic change in passenger traffic flow in the railway station; when the matching result of the composite verification identifier and the pre-stored authorization information meets the preset composite constraint condition, and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger traffic flow, activating the payment verification success state of the gate and controlling the gate to open.

[0045] The technical solution provided by this application has the following beneficial effects:

[0046] This application realizes the synchronous collection of dynamic payment credentials (QR code), biometric features (voiceprint / face) and physical media (IC card) by railway gates to ensure the comprehensiveness and real-time nature of data acquisition; enhances the anti-copying capability of payment credentials and the uniqueness of biometric features through timestamp binding and feature fusion; organically combines payment credentials, IC card identity and biometric features to construct a highly secure composite verification mark; automatically optimizes verification response time according to passenger flow density to balance security and traffic efficiency; realizes accurate identity verification through composite condition judgment to ensure the rapid passage of legitimate users.

[0047] Furthermore, the present application also parses the encrypted string from the IC card data and converts it into a fixed-length authentication code, and then arranges the dynamic token encoding sequence, authentication code and multi-dimensional biometric vector in a hierarchical structure, and combines the secure communication protocol to generate a composite verification identifier.

[0048] In addition, through standardized conversion and layered fusion, the secure integration of multi-source heterogeneous data can be achieved, which not only ensures the compatibility of IC cards with traditional payment systems, but also meets the requirements of multimodal verification for the uniformity of data structure, providing a standardized data foundation for subsequent efficient verification.

[0049] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0051] Figure 1 A flowchart of a railway diversified payment method based on multimodal recognition provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of the structure of a railway diversified payment system based on multimodal recognition provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0055] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0056] The existing railway payment system's dual verification mechanism, using static QR codes and facial recognition, suffers from three key issues: the fixed encoding of static QR codes makes them susceptible to copying and replay, and payment security relies solely on a single biometric match at a single point in time; facial recognition is affected by ambient light and facial occlusion, resulting in a high verification interruption rate; and the fixed verification duration lacks the ability to dynamically adjust, creating verification bottlenecks during peak passenger flow and redundant computing power during off-peak hours. These flaws stem from the isolated processing of verification elements and the rigid design of the system's response mechanism, making it difficult to meet the coordinated optimization requirements of payment security and transit efficiency in modern railway stations.

[0057] In response to the above limitations, the present invention proposes a railway diversified payment method based on multimodal recognition, which constructs a composite verification identification with spatiotemporal correlation through the dynamic QR code timestamp encryption binding, voiceprint-face dual biometric complementary verification and IC card identity triple collaborative authentication. Among them, the dynamic token coding sequence ensures the uniqueness of each payment, the voiceprint feature makes up for the environmental sensitivity defect of face recognition, and the dynamic adjustment mechanism of the delay threshold based on passenger flow density realizes the optimal allocation of verification resources. This method breaks through the limitation of the simple stacking of verification elements in the existing technology, and through the encrypted fusion of multimodal data and the real-time adaptation of system response parameters, it simultaneously improves the three dimensions of payment anti-counterfeiting strength, biometric fault tolerance and traffic efficiency, and completely solves the problems of easy duplication of static credentials, insufficient reliability of single biometric features and rigid system throughput.

[0058] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0059] Figure 1 A flowchart of a railway diversified payment method based on multimodal recognition is provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0060] Step 101: Collect dynamic QR code data, user real-time voice data, user face image data and IC card data at the gate in the railway station.

[0061] In step 101, dynamic QR code data is generated in real time by the railway payment system and sent to the passenger's mobile terminal. This differs from existing standard QR code data in that it embeds time-sensitive parameters (including a generation timestamp and expiration date) and a railway station identification code, and the original QR code becomes invalid immediately after each refresh. Standard QR code data, on the other hand, typically contains static, unchanging content. Real-time user voice data represents audio data collected when a passenger reads a specified string of numbers at the gate, used to extract voiceprint features. User facial image data represents a real-time facial image of the passenger captured by the gate's camera, used to extract facial geometric features. IC card data represents the encrypted identity information stored on the physical IC card used by the passenger, including the card number and issuing institution code.

[0062] In this embodiment, the gate uses a scanning module to read the dynamic QR code on the passenger's mobile phone. Simultaneously, a microphone array captures spoken voice, an infrared camera captures facial images, and a contactless card reader reads the IC card data. The dynamic QR code data is parsed to obtain encrypted payment information. The voice data undergoes noise reduction to extract the voiceprint waveform. Key point detection is used to extract contour features from the facial image. The IC card data undergoes asymmetric decryption to obtain the original card number. These four types of data are simultaneously uploaded to the central processing unit to ensure consistent collection timing.

[0063] For example, a passenger displays a dynamic QR code (containing the station logo and a 10-second validity period) on their phone at a high-speed rail station entrance gate while reading the number string "2580." The gate camera captures their facial image and reads their transportation integrated circuit card. The system simultaneously captures the encrypted QR code string, the voice waveform, the coordinates of key facial points, and the ciphertext data of the IC card, marking them as the same transaction ID.

[0064] Step 102: Generate a dynamic token encoding sequence based on the dynamic QR code data, and generate a multi-dimensional biometric feature vector based on the user's real-time voice data and the user's facial image data.

[0065] In step 102, the dynamic token code sequence represents a unique payment identifier generated based on the timestamp and gate number in the dynamic QR code, which is used to prevent replay attacks. The multi-dimensional biometric feature vector represents a feature matrix formed by fusing the spectral characteristics of the voiceprint with the coordinates of key points of the face, representing the passenger's biometric identity.

[0066] In this embodiment, the system parses the timestamp and station code in a dynamic QR code and generates a dynamic token encoding sequence through a hash operation. Voice data undergoes a Fourier transform to extract spectral envelope features, and facial images undergo a convolutional network to extract the relative positional features of facial features. These two are then concatenated according to preset weights to form a biometric vector. The dynamic token and biometric vector are bound to the same transaction ID for subsequent fusion.

[0067] For example, the dynamic QR code is parsed to obtain the timestamp "20240515143000" and the station code "BJ-S", which are then hashed with SHA-256 to generate the dynamic token "A1B2...F8". The speech spectrum is used to extract 20-dimensional MFCC coefficients, and face detection generates a 136-dimensional vector from 68 key points. These are then combined with a weighting of 0.6:0.4 to form a 156-dimensional biometric feature vector.

[0068] Step 103: extracting the identity verification code from the IC card data, performing multimodal fusion on the dynamic token encoding sequence, the identity verification code and the multi-dimensional biometric feature vector to generate a composite verification identifier.

[0069] In step 103, the identity verification code represents a fixed-length identity identifier, such as a 16-digit code, after decryption and format conversion of the IC card data. The composite verification identifier represents a data block composed of a dynamic token, an identity verification code, and a biometric vector in a hierarchical structure for unified verification.

[0070] In this embodiment, the IC card ciphertext is decrypted using RSA (Rivest-Shamir-Adleman), and the last 8 digits of the card number are truncated. These are then concatenated with the issuer code to generate the authentication code. The dynamic token serves as the header, the authentication code as the middle segment, and the biometric vector as the tail segment. These three are byte-aligned and a parity bit is appended to generate the final composite authentication identifier.

[0071] For example, after decryption, the IC card number "12345678" and the issuer code "CUP" are concatenated to form "12345678CUP." The hashed 16 digits are "5E8D…3F." The header (32-byte dynamic token), middle (16-byte authentication code), and tail (256-byte biometric vector) are concatenated, and a 3-byte check digit is appended to generate a composite verification identifier totaling 307 bytes.

[0072] Step 104: Based on the dynamic change of passenger traffic in the railway station, adjust the delay threshold between the composite verification identifier and the gate machine response action.

[0073] In step 104, the dynamic change represents the number of people passing through the gate area per unit time, as counted in real time. The delay threshold represents the maximum time allowed for the verification process. It is dynamically adjusted based on passenger flow density and is derived from dynamic traffic requirements in railway scenarios (e.g., delays may need to be shortened during peak hours and relaxed during off-peak hours).

[0074] In this embodiment, the system counts the number of passengers within the 5-meter area in front of the gate every minute and calculates the safe passage interval (total time divided by the number of people). The delay threshold is dynamically adjusted based on the inverse relationship between the complexity of the current composite verification mark (e.g., the number of layers) and the passage interval. The threshold is shortened during peak hours and relaxed during off-peak hours.

[0075] For example, if 50 people pass through per minute, the safety interval is 60 / 50, which is 1.2 seconds. If the composite verification mark complexity is level 3 (header + middle + tail), the delay threshold is 1.2 seconds × 0.8 (complexity coefficient) = 0.96 seconds, rounded to 1 second.

[0076] Step 105: When the matching result between the composite verification identifier and the pre-stored authorization information satisfies the preset composite constraint condition, and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger traffic flow, the payment verification success state of the gate is activated and the gate is controlled to open.

[0077] In step 105, the pre-stored authorization information refers to a set of valid user credentials pre-stored in the railway gate verification system. This information includes a dynamic token whitelist (a range of authorized dynamic token code sequences), an IC card identity library (a list of registered IC card authentication codes), and a biometric template library (recorded user voiceprint and facial feature data). The composite constraint condition indicates that the dynamic token must be valid, the IC card must be valid, and the biometric similarity must meet the requirements.

[0078] In this embodiment of the application, the system compares the identification to be verified with pre-stored data: the dynamic token must be within the validity period, the IC card number must be in the authorization database, and the biometric similarity must exceed the threshold. If all of these conditions are met and the current delay threshold is adapted to the passenger flow density, the gate is triggered to open.

[0079] For example, the dynamic token "A1B2...F8" is within its validity period, the IC card "12345678CUP" is on the whitelist, the comprehensive similarity between the voiceprint and the face is 92%, the current delay threshold of 1 second adapts to the passenger flow density, and the gate is opened.

[0080] This application constructs a high-security composite verification mark through the triple integration of dynamic QR code anti-counterfeiting, voiceprint-face dual biometric complementarity, and IC card identity verification; combined with the passenger flow adaptive delay adjustment mechanism, it simultaneously optimizes payment anti-counterfeiting, biometric fault tolerance and passage efficiency, achieving a balance between contactless passage at railway gates and risk prevention and control.

[0081] In order to solve the problem of collaborative processing of IC card data and biometric features in multimodal payment verification, in some embodiments, step 103: extracting the identity verification code from the IC card data, performing multimodal fusion of the dynamic token encoding sequence, the identity verification code, and the multi-dimensional biometric feature vector to generate a composite verification identifier, includes:

[0082] Step 201: performing encryption protocol parsing on the IC card data to extract an encrypted string, and converting the encrypted string into an identity verification code of a first fixed length.

[0083] In step 201, encryption protocol parsing refers to the process of decoding the ciphertext data stored on the IC card according to the railway payment system's dedicated encryption and decryption rules. The encrypted string is the original data obtained after decoding, containing information such as the card number and issuing institution. The first fixed-length authentication code is a standardized identity identifier generated by converting the encrypted string. For example, when a railway station gate reads an IC card, the system parses the encrypted data on the card to obtain 32-byte hexadecimal data of the original encrypted string. The system first extracts the first 24 bytes as the base string, then calculates the SHA-256 hash value of this base string to obtain a 64-character hash result. The first 16 characters of the hash value are then XORed with the last 16 characters to generate a 32-character intermediate string. Finally, a modulo-65536 operation is performed on this intermediate string, grouping each 8 characters into groups. The result is converted into four groups of 4-digit decimal numbers, which are then combined to form a fixed 16-bit authentication code. The divisor 65536 in the modulo operation is a system parameter determined based on the maximum concurrent processing capacity of the railway payment system.

[0084] In this embodiment, after the gate reader reads the IC card's encrypted data, it first identifies the card's encryption protocol version and asymmetrically decrypts the data using a pre-stored decryption key, obtaining a plaintext string containing the card number and the issuing institution's code. This string is then hashed and a specific segment is extracted. After adding a checksum, a fixed-length authentication code is generated, ensuring that the authentication code output by different IC cards is of uniform length.

[0085] Step 202: The dynamic token encoding sequence, the identity verification code and the multi-dimensional biometric feature vector are arranged in layers and combined with a preset secure communication protocol to generate a composite verification identifier.

[0086] In step 202, hierarchical arrangement means organizing the data of different modalities in the physical order of head, middle, and tail. The secure communication protocol is a data encapsulation standard specially designed by the railway system for multimodal data transmission and includes verification rules.

[0087] In this embodiment, the system uses a dynamic token encoding sequence as the header identifier, an identity verification code as the middle verification segment, and a multi-dimensional biometric vector as the tail verification segment, all three aligned to a preset byte length. According to the requirements of the secure communication protocol, parity bits are added to the end of each data segment, and a cyclic redundancy check code is calculated for the complete data block, ultimately generating a composite verification identifier with a hierarchical check structure.

[0088] Here's a specific example:

[0089] A passenger uses their mobile phone to display a dynamic QR code at a high-speed rail station entrance gate. The code contains the station's logo (BJ-S) and a 10-second expiration timestamp (20240515143000), while simultaneously reading the number string "2580." The gate camera captures their facial image and reads their transportation integrated circuit card. The system simultaneously captures the encrypted QR code string, voice waveform, facial key coordinates, and IC card ciphertext data, collectively labeled as transaction ID 202405151430001. The dynamic QR code data is parsed to extract the timestamp and station code, and a SHA-256 hash is used to generate a 32-byte dynamic token (A1B2C3D4E5F6G7H8I9J0K1L2M3N4O5P6). A 20-dimensional feature vector is extracted from the voice data using Mel-frequency cepstral coefficients, and a 136-dimensional feature vector is generated from the facial image using feature point detection. This vector is then combined into a 156-dimensional biometric feature vector, with a voiceprint weight of 0.6 and a facial weight of 0.4. The IC card data is decrypted using RSA to obtain the card number 12345678 and the issuer code CUP. These are then concatenated into the string 12345678CUP. The first 16 digits of the MD5 hash value (5E8D3F2A1B4C6D9E) are then used as the authentication code. The dynamic token is used as the 32-byte header identifier, the authentication code as the 16-byte middle checksum, and the biometric vector as the 256-byte tail verification segment. These are concatenated in sequence and a 3-byte CRC checksum is appended to create a composite verification identifier with a total length of 307 bytes.

[0090] In the embodiment of the present application, through standardized IC card data processing and layered fusion mechanism, the high compatibility of traditional IC card payment is retained, and secure binding with dynamic tokens and biometrics is achieved, providing a unified and tamper-proof verification data carrier for diversified railway payments.

[0091] To further improve the security and structure of multimodal data fusion, in some embodiments, step 202: hierarchically arranging the dynamic token encoding sequence, the identity verification code, and the multi-dimensional biometric feature vector, and combining them with a preset secure communication protocol to generate a composite verification identifier, includes:

[0092] Step 301: Convert the dynamic token encoding sequence into a header identifier of a second fixed length.

[0093] In step 301, the first fixed length refers to the length of the converted authentication code (e.g., 16 digits); the second fixed length specifically refers to the byte length of the header identifier (e.g., 32 bytes). These two lengths are independent of each other and are determined by the IC card data processing rules and layered arrangement rules, respectively. The second fixed-length header identifier is a data block that is the result of formatting the dynamic token encoding sequence according to the standard byte length specified by the railway payment system. It is used to uniquely identify this payment transaction. This length is pre-set based on system processing capabilities and security requirements to ensure that all dynamic tokens have a unified storage structure before integration.

[0094] In an embodiment of the present application, after the system obtains the dynamic token encoding sequence, it first checks whether its original length meets the standard, fills the insufficient part with specific characters, and cuts off the valid segment of the excess part, and then adds the version identifier and length identifier as prefixes, and finally generates a header identifier data block with a fixed byte length.

[0095] Step 302: Convert the identity verification code into an intermediate verification segment.

[0096] In step 302, the intermediate verification segment refers to a data segment formed by standardizing and encapsulating the identity verification code, and includes necessary verification information and format identifiers, and is used to ensure the integrity of the identity verification code during data transmission.

[0097] In an embodiment of the present application, after the system receives the authentication code, it adds a start flag and an end flag before and after it respectively, then calculates the checksum of the data segment and appends it to the end, and finally performs byte alignment on the complete data segment to generate an intermediate checksum segment that meets the requirements of the communication protocol.

[0098] Step 303: The multi-dimensional biometric feature vector is formed into a tail verification segment according to a preset dimensional order.

[0099] In step 303, the tail verification segment refers to a structured data block formed by reorganizing the multi-dimensional biometric feature vectors after sorting them according to feature importance, wherein each dimension corresponds to a specific storage location and weight identifier.

[0100] In an embodiment of the present application, the system performs dimensional analysis on the biometric feature vector, rearranges the data of each dimension according to the preset feature importance sorting rules, then adds a type identifier and weight coefficient to each dimension, and finally stores all the dimensional data continuously to form a tail verification segment with a clear structure.

[0101] Step 304: Based on a preset secure communication protocol, sequentially concatenate and perform redundancy verification on the header identifier, the middle verification segment, and the tail verification segment to generate a composite verification identifier.

[0102] In step 304, sequential splicing and redundancy checking refers to the process of combining the three data segments in a fixed order of head, middle, and tail, and adding additional check information to ensure the integrity of the overall data.

[0103] In an embodiment of the present application, the system first writes the header identifier, the middle check segment and the tail verification segment into the buffer in sequence, then calculates the cyclic redundancy check code of the entire buffer and appends it to the end, and finally encrypts the complete data block to generate the final composite verification identifier.

[0104] Here's a specific example:

[0105] When a passenger enters a high-speed rail station, the system generates a 32-byte dynamic token (A1B2C3D4E5F6G7H8I9J0K1L2M3N4O5P6) and a 16-bit authentication code (5E8D3F2A1B4C6D9E). According to the secure communication protocol, the system first adds a 2-byte protocol version number (01) and a 2-byte length identifier (20) to the dynamic token, generating a 36-byte header. It then adds a start marker (AA) before the authentication code, followed by an end marker (55), and calculates a checksum (2F) for this 20-byte data segment, ultimately forming a 23-byte intermediate checksum segment. The system reorders the 156-dimensional biometric feature vector based on voiceprint priority, adding a 1-byte type identifier and a 1-byte weight identifier to each dimension, generating a 364-byte tail verification segment. The three parts are concatenated in the order of a 36-byte header, a 23-byte middle, and a 364-byte tail to obtain a 423-byte data block. The CRC32 checksum 89AB12CD is calculated and appended to the block. Finally, the complete 427-byte data is encrypted using the Advanced Encryption Standard (AES), resulting in a 451-byte composite verification identifier. The CRC32 checksum is calculated using the polynomial x^32+x^26+x^23+x^22+x^16+x^12+x^11+x^10+x^8+x^7+x^5+x^4+x^2+x+1, ensuring data transmission integrity. This verification identifier will be used for subsequent gate control decisions.

[0106] In the embodiment of the present application, through strict layered arrangement and multiple verification mechanisms, the structural integrity and transmission security of multimodal data during the fusion process are ensured, providing a highly reliable unified verification data format for the railway payment system while maintaining the independent characteristics and parseability of each modal data.

[0107] To further improve the data integrity and tamper-proof capability of the composite verification identifier, in some embodiments, step 304: sequentially concatenating and performing redundancy checking on the header identifier, the middle verification segment, and the tail verification segment based on a preset secure communication protocol to generate the composite verification identifier includes:

[0108] Step 401: According to the field sequence rule defined by the secure communication protocol, the header identifier, the middle check segment and the tail verification segment are spliced ​​in a physical order from the head to the tail to generate an initial data block.

[0109] In step 401, the field order rule refers to the data block arrangement specified in the secure communication protocol, which requires a strict order: the header identifier first, the middle checksum segment in the middle, and the tail verification segment last. The initial data block refers to the intermediate data product after splicing according to this order but before the global checksum is added.

[0110] In an embodiment of the present application, the system first reads the formatted header identification data, then reads the middle check segment data, and finally reads the tail verification segment data, and writes the three parts of data into the buffer in sequence from the head to the tail to generate a continuous initial data block, which retains the original check information of each part.

[0111] Step 402: perform a joint check on the first parity bit of the header identifier, the second parity bit of the middle verification segment, and the third parity bit of the tail verification segment in the initial data block.

[0112] In step 402, joint verification involves simultaneously verifying the parity bits of the three data segments to ensure that each segment has not been tampered with before being spliced. Specifically, this involves verifying whether the first parity bit is consistent with the odd parity check rule of the binary data in the header identifier; verifying whether the second parity bit is consistent with the odd parity check rule of the encrypted string in the middle parity check segment; and verifying whether the third parity bit is consistent with the odd parity check rule of the biometric feature vector in the tail verification segment. The first parity bit, the second parity bit, and the third parity bit are verification data added to the header identifier, the middle parity check segment, and the tail verification segment, respectively, during their generation.

[0113] In an embodiment of the present application, the system extracts the first parity check bit from the header identification part of the initial data block, extracts the second parity check bit from the middle check segment part, and extracts the third parity check bit from the tail verification segment part, calculates the parity of the corresponding data parts respectively and compares them with the stored check bits. Only when all three match can the verification be determined to be passed.

[0114] Step 403: When the first parity check bit, the second parity check bit and the third parity check bit all pass verification, a global redundancy check code is appended to the end of the initial data block, and the global redundancy check code is generated by performing a modular operation based on the byte length of the header identifier, the hash value of the middle check segment and the number of characteristic dimensions of the tail verification segment.

[0115] In step 403, the global redundancy check code refers to additional check data generated based on the characteristics of the entire data block, which is used to provide a higher level of data integrity protection. The divisor used in the modular operation is related to the data characteristics to ensure the uniform distribution of the check code.

[0116] In an embodiment of the present application, after the three local check bits are verified, the system calculates the sum of the byte length of the header identifier, the hash value of the middle check segment, and the number of characteristic dimensions of the tail verification segment, and then takes the modulus of a preset large prime number, and converts the modulus operation result into a fixed-length check code and appends it to the end of the data block.

[0117] Step 404: Mark the data block after appending the global redundancy check code as a composite verification identifier.

[0118] In an embodiment of the present application, after adding a global redundancy check code at the end of the initial data block, the system marks the complete data block, sets the status identifier to verified, and records the generation timestamp, and finally outputs a composite verification identifier that can be used by the gate verification system.

[0119] Here's a specific example:

[0120] During a passenger's entry verification process at a high-speed rail station, the system generated a 36-byte header identifier containing the protocol version number 01 and a length identifier of 20, a 23-byte intermediate checksum segment with a start marker AA and an end marker 55, and a 364-byte trailer verification segment. In accordance with the secure communication protocol, the system first concatenates these three data segments, in header-to-tail order, into a 423-byte initial data block. It then verifies the parity bit 1A of the header identifier, the parity bit 2B of the intermediate checksum segment, and the parity bit 3C of the trailer verification segment. After confirming that all three match the corresponding data segments, the global verification process begins. The system then calculates the sum of the header identifier's byte length (36), the intermediate checksum segment's hash value (5E8D3F2A), and the trailer verification segment's feature dimension count (156). The hash value 5E8D3F2A, converted to decimal, is 1588221738, resulting in a sum of 36 + 1588221738 + 156 = 1588221930. Using the preset large prime number 1048583 for modular arithmetic, 1588221930 ÷ 1048583 = 1514 with a remainder of 278308, we obtain the global redundancy check code 00044044. This 4-byte check code is appended to the end of the initial data block to generate a 427-byte composite verification identifier. The global check code is calculated as (header byte length + middle segment hash value + tail dimension number) % 1048583.

[0121] In the embodiment of the present application, through the synergistic effect of the multi-level verification mechanism, the local integrity of each data segment is ensured, and anti-tampering protection of the overall data block is provided, so that the composite verification mark has both high reliability and anti-attack capabilities in the railway payment scenario, effectively supporting the security needs of diversified payments.

[0122] To further improve the accuracy and security of dynamic token and biometric generation, in some embodiments, step 102: generating a dynamic token encoding sequence based on the dynamic QR code data, and generating a multi-dimensional biometric feature vector based on the user's real-time voice data and the user's facial image data, includes:

[0123] Step 501: Bind the time-sensitive parameters in the dynamic QR code data with the device identifier of the gate in the railway station to generate a dynamic token code sequence.

[0124] In step 501, the time-sensitive parameters refer to the time-sensitive data such as the validity timestamp and generation time contained in the dynamic QR code. The railway payment system backend server embeds the dynamic QR code in real time when generating it, and contains a generation timestamp accurate to milliseconds and a preset validity period. The timestamp is taken from the atomic clock synchronization time of the payment system, and the validity period is set according to the station security level, usually in the range of 10-30 seconds, to ensure that each generated QR code is time-sensitive. The device identifier refers to the unique number of the gate device. The device identifier of the gate in the railway station is a unique code pre-assigned to each gate. It consists of three parts: the station code, the gate type code and the serial number. It is burned into the gate firmware when the device leaves the factory and registered to the railway payment center database when the system is deployed. It is used to accurately identify the physical device location where the QR code verification occurs. The dynamic token code sequence refers to a digital certificate with timeliness and device association generated by binding these two types of data.

[0125] In an embodiment of the present application, the system first parses the timestamp and validity period data in the dynamic QR code, extracts the time information accurate to milliseconds, then obtains the device number of the current gate, concatenates the time information with the device number, and performs a hash algorithm to generate a dynamic token coding sequence that is unique and time-sensitive.

[0126] Step 502: Perform waveform segmentation processing on the user's real-time voice data, extract the first characteristic component from the waveform segmentation processing result, and simultaneously perform contour reference point calibration on the user's facial image data, and extract the second characteristic component from the contour reference point calibration result.

[0127] In step 502, waveform segmentation refers to the process of dividing a continuous speech signal into multiple analysis units based on fixed time windows. The first feature component refers to the set of voiceprint features extracted from the speech waveform. Contour reference point calibration refers to the process of locating key feature points on a facial image. The second feature component refers to the set of facial geometric features calculated based on these feature points.

[0128] In this embodiment of the present application, the system performs frame processing on the collected voice signal. After filtering and spectral analysis of each frame, the voiceprint feature parameters are extracted to form the first feature component. Simultaneously, feature point detection is performed on the facial image to locate the coordinates of key areas such as the eyes, nose, and mouth. The distances and angles between these points are calculated to form the second feature component.

[0129] Step 503: Superimpose the first feature component and the second feature component according to a preset weight ratio to generate a multi-dimensional biometric feature vector.

[0130] In step 503, the preset weight ratio refers to a weighting coefficient set according to the importance of voiceprint features and facial features in verification.

[0131] In an embodiment of the present application, the system multiplies the first feature component and the second feature component by the corresponding weight coefficients respectively, and then concatenates the weighted feature values ​​in dimensional order to form a new feature vector, which contains both voiceprint and facial feature information as multi-dimensional data representing the user's biometric characteristics.

[0132] Here's a specific example:

[0133] A passenger used their mobile phone to display a dynamic QR code at a high-speed rail station entrance gate. The code contained the timestamp 20240515143000 and the station code BJ-S, while simultaneously reading the digit string 2580. The gate camera captured their facial image. The system first parsed the dynamic QR code, extracting the timestamp 20240515143000 and the gate device number BJ-S-G05, concatenating them into the string 20240515143000BJ-S-G05. This was then hashed with a SHA-256 algorithm to generate a 32-byte dynamic token, A1B2C3D4E5F6G7H8I9J0K1L2M3N4O5P6. The captured 2-second speech signal was framed into 40 frames, with 20-dimensional Mel-frequency cepstral coefficients extracted from each frame to form an 800-dimensional first feature component. Sixty-eight feature points were detected in the facial image, resulting in a 136-dimensional second feature component. According to the ratio of 0.6 for voiceprint weight and 0.4 for face weight, the first feature component is multiplied by 0.6 to obtain a 480-dimensional weighted feature, and the second feature component is multiplied by 0.4 to obtain a 54-dimensional weighted feature. These are then concatenated to generate a 534-dimensional multi-dimensional biometric feature vector. The weights of 0.6 and 0.4 are determined based on the ratio of 95% voiceprint recognition accuracy to 90% face recognition accuracy in railway scenarios. The calculation formula is: voiceprint weight = voiceprint accuracy / (voiceprint accuracy + face accuracy) = 0.95 / 1.85 ≈ 0.6, and face weight = 1-0.6 = 0.4. This biometric feature vector, together with the dynamic token and IC card authentication code, constitutes the basic data for subsequent verification.

[0134] In the embodiment of the present application, the uniqueness of the dynamic token is ensured by dual binding of time and device, and the characterization capability of biometric features is improved by weighted fusion of voiceprint and facial features, providing high-security and high-accuracy identity authentication basic data for the railway payment system.

[0135] To further improve the dynamic adaptability of gate response efficiency and security, in some embodiments, step 104: adjusting the delay threshold between the composite verification identifier and the gate response action based on the dynamic change in passenger flow in the railway station, includes:

[0136] Step 601: Real-time monitoring of passenger density parameters in the gate area of ​​a railway station, determining a dynamic change in passenger flow rate based on the passenger density parameters, and calculating a safe passage interval between passengers based on the dynamic change.

[0137] In step 601, the passenger density parameter refers to the statistical value of the number of passengers passing through the gate area per unit time. The safe passage interval refers to the minimum time interval required to ensure the safe passage of passengers, which is inversely proportional to the passenger density.

[0138] In an embodiment of the present application, the system uses smart cameras deployed in the gate area to count the number of passengers passing through the designated area per minute in real time, establishes a dynamic change curve based on historical traffic data, and calculates the safety interval time that each passenger should maintain through a formula based on the current density value. This time decreases as the density increases.

[0139] Step 602: Based on the relationship between the hierarchical structure complexity of the composite verification identifier and the inverse of the safe passage interval, dynamically adjust the delay threshold, wherein if the adjustment direction is to reduce the delay threshold, it means shortening the verification time window of the head identifier and the tail verification segment in the composite verification identifier, so that the delay threshold is negatively correlated with the safe passage interval.

[0140] In step 602, the layered structure complexity refers to the data structure complexity of the header identifier, the middle check segment, and the tail verification segment in the composite verification identifier. The negative correlation refers to the adjustment mechanism that the delay threshold is shortened as the safety interval decreases.

[0141] In an embodiment of the present application, the system analyzes the byte length and verification level of each data segment in the composite verification identifier, evaluates its overall complexity level, multiplies the level by the inverse of the real-time safe passage interval, and obtains a delay threshold adapted to the current passenger flow status. When the passenger flow density increases, the verification time window is automatically shortened to ensure passage efficiency. For example, during peak hours at a railway station, when the passenger density reaches 50 people per minute (real-time statistics by cameras in the gate area), the system calculates the safe passage interval to be 1.2 seconds (= 60 seconds / 50 people); at this time, the hierarchical structure complexity of the composite verification identifier is level 3 (calculated based on the total bit weight of the 32-bit header identifier, 16-bit middle check segment, and 256-bit tail verification segment). The system dynamically adjusts the verification timeout to 0.7 seconds (retaining 1 decimal place) according to the formula: delay threshold (seconds) = normalized value of (inverse of safe passage interval × complexity coefficient) = (1 / 1.2×0.8) ≈ 0.67 seconds, where the complexity coefficient of 0.8 is the optimal efficiency parameter under level 3 complexity measured experimentally.

[0142] Here's a specific example:

[0143] During the morning rush hour at a high-speed rail station, the system detected 75 passengers waiting to pass per minute using 3D sensing equipment deployed in the gate area. Based on the safe clearance interval calculation formula (60 seconds divided by 75 passengers), the current safe clearance interval is 0.8 seconds. The system processes a composite verification token consisting of a 32-byte header, a 16-byte middle checksum, and a 256-byte tail verification token. Analysis by the complexity assessment module determined its structural complexity level to be Level 3. The system uses a preset complexity coefficient comparison table, and the adjustment coefficient for Level 3 complexity is 0.7. Multiplying the 0.8-second safe clearance interval by the complexity coefficient of 0.7 yields a theoretical delay threshold of 0.56 seconds. Considering the gate motor's minimum response time of 0.1 seconds, the system rounds the delay threshold to 0.6 seconds. Based on this threshold, the system dynamically adjusts the time allocation for each step in the verification process: the header verification time is shortened to 0.15 seconds, the middle checksum remains at 0.2 seconds, and the tail verification is compressed to 0.25 seconds. When the dynamic token A1B2C3D4E5F6G7H8I9J0K1L2M3N4O5P6 is detected to be within its validity period and the IC card 5E8D3F2A1B4C6D9E is verified, the system completes the entire verification process within 0.55 seconds, and the biometric similarity reaches 91%, meeting the 0.6-second delay threshold requirement, and then triggers the gate opening instruction. The complexity coefficient of 0.7 is based on historical data statistical analysis and is calculated as complexity coefficient = 1 / (1+ln(number of levels)). Level 3 complexity corresponds to ln(3)≈1.0986, so the coefficient is 1 / 2.0986≈0.4765, and the safety factor is multiplied by 1.5 to determine 0.7.

[0144] In the embodiment of the present application, an intelligent balance between gate response speed and safety interval is achieved through dynamic coupling adjustment of passenger flow density and verification complexity, which not only avoids queue congestion during peak hours, but also ensures the reliability of the verification process, enabling the railway payment system to have the ability to adapt to changes in passenger flow.

[0145] To further improve the accuracy and response efficiency of payment verification, in some embodiments, step 105: when the matching result between the composite verification identifier and the pre-stored authorization information satisfies the preset composite constraint condition, and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger passage flow, activating the payment verification success state of the gate and controlling the gate to open includes:

[0146] Step 701: Match the header identifier, the middle check segment and the tail verification segment in the composite verification identifier with the dynamic token whitelist, the IC card identity library and the biometric template in the pre-stored authorization information segment by segment.

[0147] In step 701, the dynamic token whitelist contains all valid, unused dynamic token records. The IC card identity library stores registered, legitimate IC card information. The biometric template library stores pre-recorded user voiceprint and facial feature data. Segment-by-segment matching is the process of comparing and verifying each component of a composite verification identifier with pre-stored authorization information of the corresponding type.

[0148] In an embodiment of the present application, the system first disassembles the composite verification identifier into a hierarchical structure, verifies the timeliness and uniqueness of the header identifier against the records in the dynamic token whitelist, matches the middle verification segment with the card number and issuer information in the IC card identity library, and calculates the similarity between the tail verification segment and the feature data in the biometric template library.

[0149] Step 702: When the header identifier exists in the dynamic token whitelist, the middle verification segment matches any record in the IC card identity database, and the similarity between the tail verification segment and the biometric template exceeds a preset ratio, it is determined that the composite constraint condition is met.

[0150] In step 702, the preset ratio refers to the minimum similarity requirement for biometric verification to pass, which is dynamically adjusted according to the security level. The composite constraint refers to the three verification criteria that need to be met simultaneously, including the validity of the dynamic token, the legitimacy of the IC card, and the biometric matching degree.

[0151] In this embodiment of the application, the system sequentially checks whether the header identifier is in the valid token list, whether the middle verification segment matches any valid IC card record, and whether the tail verification segment is similar to the template. Only when all three verifications are passed will the composite constraint condition be determined to be met and the next stage of detection will be entered.

[0152] Step 703: Synchronously detect whether the current value of the adjusted delay threshold is within the allowable range corresponding to the safe passage interval. If it is satisfied, send an opening instruction to the gate control unit and mark the current payment verification status as successfully activated.

[0153] In step 703, the allowable interval refers to a reasonable range of delay thresholds calculated based on real-time passenger flow density, which is used to ensure a balance between verification speed and traffic safety. The successful activation status refers to the transaction success indicator recorded by the system after the payment verification is passed.

[0154] In this embodiment of the application, after the composite constraint conditions are met, the system checks whether the current delay threshold is within a safe interval calculated based on passenger flow density. If so, it sends an opening signal to the gate and simultaneously records the successful verification status and precise timestamp in the transaction log, completing the entire payment verification process.

[0155] Here's a specific example:

[0156] When a passenger is verifying entry into a high-speed rail station, the system has generated a composite verification identifier consisting of a header identifier A1B2C3D4E5F6G7H8I9J0K1L2M3N4O5P6, an intermediate verification segment 5E8D3F2A1B4C6D9E, and a 156-dimensional biometric vector for the tail verification segment. The system first queries the dynamic token whitelist database to confirm that the timestamp 20240515143000 of the header identifier A1B2C3D4E5F6G7H8I9J0K1L2M3N4O5P6 is within the validity period and has not been used. It then searches the IC card identity database to find the legal card number 12345678CUP corresponding to the intermediate verification segment 5E8D3F2A1B4C6D9E. The similarity between the tail verification segment and the pre-stored biometric template is then calculated using the cosine similarity formula. ; Among them, the verification vector The first biometric feature vector to be verified dimensional eigenvalue; template vector Indicates the first dimensional eigenvalues; Represents the total number of dimensions of the feature vector (e.g., 156). The resulting similarity is 92%, exceeding the preset 90% standard. The system detects that 50 people pass through the current gate area per minute. According to the formula, safe interval = 60 seconds / 50 people = 1.2 seconds, and the allowable delay threshold range is 0.9-1.5 seconds. The actual delay threshold of 1 second falls within this range. The system then sends an opening command to the gate control unit and records the successful verification in the transaction log, marking transaction ID T202405151430001 as verified. The entire verification process takes 0.95 seconds, including 0.3 seconds for the header identifier verification, 0.2 seconds for the middle verification segment, and 0.45 seconds for the tail verification segment match. This meets the 1-second delay threshold requirement, and the gate opens smoothly.

[0157] In the embodiment of the present application, through the strict control of triple verification conditions and the intelligent adaptation of delay thresholds, payment security and identity authenticity are ensured, and dynamic optimization of traffic efficiency is achieved, providing railway passengers with a safe and convenient contactless payment experience, while effectively preventing ticket fraud and illegal intrusion risks.

[0158] Figure 2 A schematic diagram of the structure of a railway diversified payment system based on multimodal recognition provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the system includes:

[0159] The collection module 21 is used to collect dynamic QR code data, user real-time voice data, user face image data and IC card data at the gate in the railway station.

[0160] The generating module 22 is configured to generate a dynamic token encoding sequence based on the dynamic two-dimensional code data, and to generate a multi-dimensional biometric feature vector based on the user's real-time voice data and the user's facial image data.

[0161] The extraction module 23 is used to extract the identity verification code from the IC card data, and perform multimodal fusion of the dynamic token encoding sequence, the identity verification code and the multi-dimensional biometric feature vector to generate a composite verification mark.

[0162] The adjustment module 24 is used to adjust the delay threshold between the composite verification mark and the gate machine response action based on the dynamic change of the passenger traffic flow in the railway station.

[0163] The activation module 25 is used to activate the payment verification success status of the gate and control the gate to open when the matching result of the composite verification identifier and the pre-stored authorization information meets the preset composite constraint conditions and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger traffic flow.

[0164] Figure 2 The railway diversified payment system based on multimodal recognition can be implemented Figure 1 The implementation principles and technical effects of the multimodal recognition-based railway diversified payment method described in the illustrated embodiment are not further elaborated. The specific manner in which each module and unit performs operations in the multimodal recognition-based railway diversified payment system described in the above embodiment has been described in detail in the related embodiments and will not be further elaborated here.

[0165] In one possible design, Figure 2 The railway diversified payment system based on multimodal recognition of the embodiment shown can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0166] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0167] The processing component 32 is as follows Figure 1 The embodiment provides a railway diversified payment method based on multimodal recognition.

[0168] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0169] The storage component 31 is configured to store various types of data to support operations on the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0170] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0171] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0172] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0173] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0174] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a railway diversified payment method based on multimodal recognition.

[0175] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0176] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0177] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A railway diversified payment method based on multimodal recognition, characterized in that: include: Collect dynamic QR code data, user real-time voice data, user face image data, and IC card data at the gate in the railway station; Generate a dynamic token encoding sequence based on the dynamic QR code data, and generate a multi-dimensional biometric feature vector based on the user's real-time voice data and the user's facial image data; Extracting the identity verification code from the IC card data, performing multimodal fusion on the dynamic token encoding sequence, the identity verification code and the multi-dimensional biometric feature vector to generate a composite verification identifier; Adjusting the delay threshold between the composite verification mark and the gate's response action based on the dynamic change in passenger flow within the railway station; When the matching result between the composite verification identifier and the pre-stored authorization information meets the preset composite constraint conditions, and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger traffic flow, the payment verification success status of the gate is activated and the gate is controlled to open.

2. The method according to claim 1, characterized in that The step of extracting the identity verification code from the IC card data and performing multimodal fusion of the dynamic token encoding sequence, the identity verification code, and the multi-dimensional biometric feature vector to generate a composite verification identifier includes: Performing encryption protocol parsing on the IC card data to extract an encrypted string, and converting the encrypted string into an identity verification code of a first fixed length; The dynamic token encoding sequence, the identity verification code and the multi-dimensional biometric feature vector are arranged in layers and combined with a preset secure communication protocol to generate a composite verification mark.

3. The method according to claim 2, characterized in that The step of arranging the dynamic token encoding sequence, the identity verification code, and the multi-dimensional biometric feature vector in layers and combining them with a preset secure communication protocol to generate a composite verification identifier includes: Converting the dynamic token encoding sequence into a header identifier of a second fixed length; Converting the identity verification code into an intermediate verification segment; Forming a tail verification segment of the multi-dimensional biometric feature vector according to a preset dimensional order; Based on a preset secure communication protocol, the header identifier, the middle verification segment, and the tail verification segment are sequentially spliced ​​and redundantly verified to generate a composite verification identifier.

4. The method according to claim 3, characterized in that The step of sequentially splicing and performing redundancy checking on the header identifier, the middle verification segment, and the tail verification segment based on a preset secure communication protocol to generate a composite verification identifier includes: According to the field sequence rule defined by the secure communication protocol, the header identifier, the intermediate check segment, and the tail verification segment are spliced ​​in a physical order from head to tail to generate an initial data block; Performing a joint check on a first parity check bit of a header identifier, a second parity check bit of a middle verification segment, and a third parity check bit of a tail verification segment in the initial data block; When the first parity check bit, the second parity check bit, and the third parity check bit all pass verification, appending a global redundancy check code to the end of the initial data block, where the global redundancy check code is generated by performing a modular operation based on the byte length of the header identifier, the hash value of the middle check segment, and the number of characteristic dimensions of the tail verification segment; The data block after the global redundancy check code is added is marked as a composite verification identifier.

5. The method according to claim 1, wherein The method of generating a dynamic token encoding sequence based on the dynamic two-dimensional code data and generating a multi-dimensional biometric feature vector based on the user's real-time voice data and the user's facial image data includes: Binding the time-sensitive parameters in the dynamic QR code data with the device identifier of the gate in the railway station to generate a dynamic token code sequence; Performing waveform segmentation processing on the real-time voice data of the user, extracting a first characteristic component from the waveform segmentation processing result, and performing contour reference point calibration on the facial image data of the user, and extracting a second characteristic component from the contour reference point calibration result; The first feature component and the second feature component are superimposed according to a preset weight ratio to generate a multi-dimensional biometric feature vector.

6. The method according to claim 1, characterized in that The method of adjusting the delay threshold between the composite verification identifier and the gate machine response action based on the dynamic change of the passenger flow in the railway station includes: Real-time monitoring of passenger density parameters in the gate area of ​​a railway station, determining a dynamic change in passenger flow based on the passenger density parameters, and calculating a safe passage interval between passengers based on the dynamic change; Based on the relationship between the hierarchical structure complexity of the composite verification identifier and the inverse of the safe passage interval, the delay threshold is dynamically adjusted. If the adjustment direction is to reduce the delay threshold, it means shortening the verification time window of the head identifier and the tail verification segment in the composite verification identifier so that the delay threshold is negatively correlated with the safe passage interval.

7. The method according to claim 1, characterized in that When the matching result between the composite verification identifier and the pre-stored authorization information satisfies the preset composite constraint condition, and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger passage flow, activating the payment verification success state of the gate and controlling the gate to open, including: Match the header identifier, the middle check segment and the tail verification segment in the composite verification identifier with the dynamic token whitelist, the IC card identity library and the biometric template in the pre-stored authorization information segment by segment; When the header identifier exists in the dynamic token whitelist, the intermediate verification segment matches any record in the IC card identity database, and the similarity between the tail verification segment and the biometric template exceeds a preset ratio, it is determined that the composite constraint condition is satisfied; Synchronously detect whether the current value of the adjusted delay threshold is within the allowable range corresponding to the safe passage interval. If it is satisfied, send an opening instruction to the gate control unit and mark the current payment verification status as successfully activated.

8. A railway diversified payment system based on multimodal recognition, characterized by: include: The acquisition module is used to collect dynamic QR code data, user real-time voice data, user face image data and IC card data at the gate in the railway station; A generation module, configured to generate a dynamic token encoding sequence based on the dynamic QR code data, and generate a multi-dimensional biometric feature vector based on the user's real-time voice data and the user's facial image data; An extraction module, configured to extract the identity verification code from the IC card data, and perform multimodal fusion of the dynamic token encoding sequence, the identity verification code, and the multi-dimensional biometric feature vector to generate a composite verification identifier; An adjustment module, configured to adjust a delay threshold between the composite verification identifier and a gate response action based on a dynamic change in passenger flow within the railway station; The activation module is used to activate the payment verification success status of the gate and control the gate to open when the matching result of the composite verification identifier and the pre-stored authorization information meets the preset composite constraint conditions and the adjusted delay threshold is consistent with the safe passage interval corresponding to the passenger traffic flow.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a railway diversified payment method based on multimodal recognition as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, a railway diversified payment method based on multimodal recognition as described in any one of claims 1 to 7 is implemented.

Images