Subway station-gate data mismatch identification method based on tensor decomposition

Abstract

This invention proposes a station-gate mismatch information identification model combining tensor decomposition and neural networks. First, tensor decomposition is used to extract the associated passenger flow features of each gate. Then, a balanced feature dataset is generated through manual oversampling as training data for the neural network. The neural network performs binary classification through cross-validation of the features, thereby identifying potential mismatched gates in the AFC data. Results show that the model can identify gate-station mismatch with over 90% accuracy. For mismatched gates, the model can identify them with over 90% accuracy, while the false recognition rate for matched gates is very low. Especially when the mismatch rate reaches 40%, the accuracy remains above 90%, demonstrating the model's strong robustness.

Description

Technical Field

[0001] This invention belongs to the field of transportation big data, specifically a method for identifying data mismatch between subway stations and turnstiles based on tensor decomposition. Background Technology

[0002] Automated Fare Collection (AFC) system transaction data is one of the most important data sources for public transportation. In an AFC system, when a passenger passes through a ticket gate, the system retrieves the corresponding gate's associated station information to determine the origin and destination of the journey. Station-gate mismatch refers to one or more gates being incorrectly recorded as belonging to a different station in the AFC system database. For example, gate #001, located at station A in a subway system, might be incorrectly recorded as being located at station B. Station-gate mismatch can cause errors in distance-based fare calculation systems, and if left undetected for an extended period, it can result in significant economic losses.

[0003] Among existing Chinese patent research, Chinese Patent 201910629489.1 uses deep reinforcement learning technology to build a model for microscopic traffic simulation environments, significantly improving the effectiveness of traffic signal control in complex traffic conditions. Chinese Patent 202310436439.8 optimizes the initial operation plan of urban rail trains based on AFC passenger flow data and parameters such as the relationship table between weighted passenger flow time periods and weights, to formulate the final operation plan, effectively improving the connection between urban rail transit and high-speed rail. Chinese Patent 202310295408.5 uses the K-short-circuit algorithm combined with AFC card swiping data to predict passenger flow in the station's paid area, greatly avoiding large-scale stampedes in the paid area. Chinese Patent 202310103624.5 simulates the function of AFC terminal equipment through gate drive units and other components, and detects faults in AFC terminal equipment through a computer detection unit, realizing the detection of AFC terminal equipment function and greatly shortening the detection cycle. Overall, existing research tends to focus on the application of AFC data, and there is still a lack of effective methods for addressing the AFC data quality problem of site-gate information mismatch. Summary of the Invention

[0004] To overcome the shortcomings of the existing technology, the present invention aims to propose a model combining tensor decomposition and neural networks to identify turnstiles with incorrect site information in the database, thereby solving the problem of site-turntile information mismatch. Tensor decomposition technology is used to extract the spatiotemporal characteristics of passenger flow through each turnstile, and a neural network cross-validation method is used to identify mismatched turnstiles.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A method for identifying data mismatch between subway stations and turnstiles based on tensor decomposition, specifically including the following steps:

[0007] Step 1: Obtain AFC data from the subway system and preprocess it;

[0008] The gate ID is converted into the corresponding station name using the station-gate matching table. The preprocessed AFC data includes five fields: smart card ID, entry time, entry station, exit time, and exit station.

[0009] Step 2: Extract the associated passenger flow of the turnstiles to form an associated passenger flow dataset;

[0010] The AFC data records passing through gate m are aggregated to obtain all entry / exit records of the subway system entering (entry gate) / leaving (exit gate) through gate m, forming the associated passenger flow dataset of gate m;

[0011] Step 3: Construct the associated passenger flow tensor;

[0012] Passenger flow is counted according to four dimensions: “gate dimension (M)”, “time dimension (T)”, “date dimension (D)”, and “site dimension (S)”. By digitizing the coordinates of each dimension, the passenger flow associated with the gates is constructed as a 4th-order tensor with dimensions M×T×D×S.

[0013] Step 4: Perform CP decomposition on the associated passenger flow tensor;

[0014] Step 5: Extract the associated passenger flow feature vector;

[0015] Extract the factor vectors of each gate to form a feature matrix. The row represents the associated passenger flow feature vector of the corresponding gate.

[0016] Step 6: Construct the site neural network;

[0017] For each station S i Construct a neural network N corresponding to this site. i =f(W i X i +b i ), where N i This is the output of the site's neural network, f is the activation function, and W is the output of the site's neural network. i It is the weight matrix, X i It is the input feature vector, b i This is the bias vector. The network input is the associated passenger flow feature vector of all turnstiles, and the output is the probability that the input turnstile matches the station. For the station neural network N... i If the station record of gate m is S iIf the condition is met, the associated passenger flow feature vector of gate m is labeled as 1; otherwise, it is labeled as 0. In each training iteration, all feature vectors labeled as 0 and n-1 feature vectors labeled as 1 are selected as training samples.

[0018] Step 7: Balance the dataset using the ADASYN method;

[0019] Calculate the imbalance based on the number of positive and negative samples: Where m + m is the number of positive samples. - The number of negative samples; if d is less than d th (The preset maximum class imbalance) represents the minority class samples (here, the positive examples). Calculate the number of data samples needed to synthesize: G = (m - -m + )×β, where β is used to control the balance level; for each sample x in the minority class i Calculate the ratio and normalize it: Where Δ i It is x i The number of samples belonging to the negative class among x's k nearest neighbors; calculate the number of samples belonging to the negative class among x's k nearest neighbors. i Number of samples to be generated: Thus, artificial positive sample x is obtained. i,j =x i +λ(x zi -x i ), where x zi It is x i One of the nearest neighbors, where λ is a random number. Using this ADASYN method, m features similar to those of positive examples are generated. - -m + To obtain artificial positive examples, the dataset is brought to an approximately balanced state, thus eliminating learning bias.

[0020] Step 8: Train the site's neural network to identify mismatched turnstiles;

[0021] For each site's neural network, the oversampled turnstile-associated passenger flow feature vector sample set is cross-validated and trained using the method described in step 6. After each training iteration, the resulting network is used to validate one of the turnstiles. If the output probability of a turnstile is lower than the threshold α, it is determined to be a mismatched turnstile; otherwise, it is determined to be a match.

[0022] Furthermore, step 4 specifically involves:

[0023] For the correlation tensor V∈R M×T×D×S Its CP decomposition takes the form of: Where R represents the number of factors, m r ∈R M , t r∈R T d r ∈R D s r ∈R S These represent factor vectors for the turnstile, time, date, and site dimensions, respectively. Represents the vector outer product; the factor vectors are combined to form a factor matrix M = [m 1 ,m 2 ,…,m R ]∈R M×R T = [t 1 ,t 2 ,…,t R ]∈R T×R D = [d 1 ,d 2 ,…,d R ]∈R D×R S = [s 1 ,s 2 ,…,s R ]∈R S×R Expanding the associated passenger flow tensor into a matrix in 4 dimensions, we have V (1) =M(T⊙D⊙S) T V (2) =T(M⊙D⊙S) T V (3) =D(M⊙T⊙S) T V (4) =S(M⊙T⊙D) T Where ⊙ denotes the Khatri-Rao product; keeping other factor matrices unchanged, update the factor matrix M and solve for m. r Make Minimize, that is, find the solution Among them, ||…|| F V represents the Frobenius norm. (1) ∈R M×TDS It is the matrix formed by expanding the associated passenger flow tensor along the gate dimension, (T⊙D⊙S)∈R R ×TDS Let V represent the Khatri-Rao product of the other factor matrices; update the factor matrices T, D, and S sequentially according to the above steps to minimize the approximate error between the associated passenger flow tensor and its CP decomposition tensor, and obtain the approximate tensor V after CP decomposition.

[0024] Furthermore, step 6 specifically includes:

[0025] For each site S i Construct a neural network N corresponding to this site. i =f(W i X i +bi ), where N i This is the output of the site's neural network, f is the activation function, and W is the output of the site's neural network. i It is the weight matrix, X i It is the input feature vector, b i This is the bias vector. The network input is the associated passenger flow feature vector of all turnstiles, and the output is the probability that the input turnstile matches the station. For the station neural network N... i If the station record of gate m is S i If the condition is met, the associated passenger flow feature vector of gate m is labeled as 1; otherwise, it is labeled as 0. In each training iteration, all feature vectors labeled as 0 and n-1 feature vectors labeled as 1 are selected as training samples.

[0026] Furthermore, step 7 specifically includes:

[0027] Calculate the imbalance based on the number of positive and negative samples: Where m + m is the number of positive samples. - The number of negative samples; if d is less than d th (The preset maximum class imbalance) represents the minority class samples (here, the positive examples). Calculate the number of data samples needed to synthesize: G = (m - -m + )×β, where β is used to control the balance level; for each sample x in the minority class i Calculate the ratio and normalize it: Where Δ i It is x i The number of samples belonging to the negative class among x's k nearest neighbors; calculate the number of samples belonging to the negative class among x's k nearest neighbors. i Number of samples to be generated: Thus, artificial positive sample x is obtained. i,j =x i +λ(x zi -x i ), where x zi It is x i One of the nearest neighbors, where λ is a random number. Using this ADASYN method, m features similar to those of positive examples are generated. - -m + We use artificial positive samples to bring the dataset to an approximately balanced state and eliminate learning bias.

[0028] Compared to existing technologies, this application offers the following advantages: The proposed method for identifying subway station-turnstile data mismatch based on tensor decomposition utilizes tensor decomposition to reduce data dimensionality and complexity, and employs a neural network model for turningstile mismatch identification. The model exhibits extremely high accuracy in identifying the station affiliation of turnstiles. When the mismatch ratio is 0.05, the accuracy rate for identifying mismatched turnstiles reaches 97.22%, and even at a mismatch ratio of 0.4, it maintains an accuracy rate of 92.4%. The model reliably extracts and identifies the associated passenger flow characteristics of the turnstiles, effectively reducing losses caused by data quality issues such as station-turnstile information mismatch, and significantly reducing the manpower and economic costs of verifying and checking AFC data. Attached Figure Description

[0029] Figure 1 This is a flowchart of a method according to an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram of tensor construction according to an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of tensor decomposition according to an embodiment of the present invention. Detailed Implementation

[0032] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0033] like Figure 1 As shown in the figure, this embodiment provides a method for identifying data mismatch between subway stations and turnstiles based on tensor decomposition, which specifically includes the following steps:

[0034] Step 1: Obtain AFC card swipe data for Guangzhou Metro from January 8th to January 14th, 2017 (7 days). During preprocessing, five fields are selected: Smart Card ID, Entry Time, Entry Gate ID, Exit Time, and Exit Gate ID. Simultaneously, a station-gate matching table is used to convert the gate ID to the corresponding station name. The preprocessed AFC data includes the following five fields: Smart Card ID, Entry Time, Entry Station, Exit Time, and Exit Station.

[0035] Step 2: For each entrance and exit gate in this dataset, extract all entrance / exit records of the subway system through the gates to construct a related passenger flow dataset.

[0036] Step 3: Construct a gate-related passenger flow tensor. This tensor includes four dimensions: "gate dimension," "time dimension," "date dimension," and "site dimension." By digitizing the coordinates of each dimension, the passenger flow can be represented by a 4th-order tensor with two dimensions: 1247×24×7×91 (entry passenger flow) and 1191×24×7×91 (exit passenger flow). The structure is as follows: Figure 2 .

[0037] Step 4: Process the associated passenger flow tensor Perform tensor CP decomposition, the structure is as follows Figure 3 The parameters of the CP decomposition algorithm are shown in Table 1.

[0038] Step 5: Extract the feature vector matrix of the gate dimension after CP decomposition. Each row represents the passenger flow characteristics of a turnstile.

[0039] Step 6: For each site S i Construct a neural network N corresponding to this site. i The specific parameter settings for the neural network model are shown in Table 1. For the site neural network N... i If the station record of gate m is S i If the condition is met, the associated passenger flow feature vector of gate m is labeled as 1; otherwise, it is labeled as 0. All associated passenger flow feature vectors labeled as 1 are divided into 5 equal parts, and 5 cross-validations are performed to identify mismatched gates. In each training iteration, all feature vectors labeled as 0 and 4 feature vectors labeled as 1 are selected as training samples.

[0040] Table 1 Model Parameter Settings

[0041]

[0042]

[0043] Step 7: Use the ADASYS algorithm to oversample the positive samples so that the ratio of positive to negative samples in the training data of each site network is 1:1.

[0044] Step 8: For each site neural network, the oversampled gate-associated passenger flow feature vector sample set is cross-validated and trained using the method described in Step 6. After each training, the obtained network is used to verify one of the gates. If the output probability of the gate is less than 80%, it is determined to be a mismatched gate; otherwise, it is determined to be a match.

[0045] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention, and all such changes are within the protection scope of the technology.

Claims

1. A method for identifying data mismatch between subway station and turnstiles based on tensor decomposition, characterized in that, Includes the following steps: Step 1: Obtain AFC data from the subway system and preprocess it; The gate ID is converted into the corresponding station name using the station-gate matching table. The preprocessed AFC data includes five fields: smart card ID, entry time, entry station, exit time, and exit station. Step 2: Extract the associated passenger flow of the turnstiles to form an associated passenger flow dataset; The AFC data records passing through gate m are aggregated to obtain all entry / exit records of the subway system through gate m, forming the associated passenger flow dataset of gate m; Step 3: Construct the associated passenger flow tensor; Passenger flow is counted according to four dimensions: "gate dimension M", "time dimension T", "date dimension D" and "site dimension S". By digitizing the coordinates of each dimension, the passenger flow associated with the gate is constructed into a 4th order tensor V with dimensions M×T×D×S. Step 4: Perform CP decomposition on the associated passenger flow tensor; Step 5: Extract the associated passenger flow feature vector; Extract the factor vectors of each gate to form a feature matrix. The row represents the associated passenger flow feature vector of the corresponding gate. Step 6: Construct the site neural network; For each station S i Construct a neural network N corresponding to this site. i =f(W i X i +b i ), where N i This is the output of the site's neural network, f is the activation function, and W is the output of the site's neural network. i It is the weight matrix, X i It is the input feature vector, b i It is a bias vector; the network input is the associated passenger flow feature vector of all turnstiles, and the output is the probability that the input turnstile matches the station; for a station, the neural network N i If the station record of gate m is S i If the associated passenger flow feature vector of gate m is 1, then it is labeled as 0; otherwise, it is labeled as 0. In each training, all feature vectors labeled as 0 and n-1 feature vectors labeled as 1 are selected as training samples. Step 7: Balance the dataset using the ADASYN method; Step 8: Train the site's neural network to identify mismatched turnstiles; For each site's neural network, the oversampled gate-associated passenger flow feature vector sample set is cross-validated and trained using the method in step 6. After each training, the obtained network is used to validate one of the gates. If the output probability of the gate is lower than the threshold α, it is determined to be a mismatched gate; otherwise, it is determined to be a match.

2. The method for identifying subway station-turnstile data mismatch based on tensor decomposition as described in claim 1, characterized in that, Step 4 is as follows: For the correlation tensor V∈R M×T×D×S Its CP decomposition takes the form of: Where R represents the number of factors, m r ∈R M , t r ∈R T d r ∈R D s r ∈R S These represent factor vectors for the turnstile, time, date, and site dimensions, respectively. Represents the vector outer product; the factor vectors are combined to form a factor matrix M = [m 1 ,m 2 ,…,m R ]∈R M×R , T = [t 1 ,t 2 ,…,t R ]∈R T×R D = [d 1 ,d 2 ,…,d R ]∈R D×R S = [s 1 ,s 2 ,…,s R ]∈R S×R Expanding the associated passenger flow tensor into a matrix in 4 dimensions, we have V (1) =M(T⊙D⊙S) T V (2) =T(M⊙D⊙S) T V (3) =D(M⊙T⊙S) T V (4) =S(M⊙T⊙D) T Where ⊙ denotes the Khatri-Rao product; keeping other factor matrices unchanged, update the factor matrix M and solve for m. r Make Minimize, that is, find the solution Among them, ||…|| F V represents the Frobenius norm. (1) ∈R M×TDS It is the matrix formed by expanding the associated passenger flow tensor along the gate dimension, (T⊙D⊙S)∈R R ×TDS Represent the Khatri-Rao product of other factor matrices; update factor matrices T, D, and S sequentially according to the above steps to minimize the approximate error between the associated passenger flow tensor and its CP decomposition tensor, and obtain the approximate tensor after CP decomposition.

3. The method for identifying subway station-turnstile data mismatch based on tensor decomposition as described in claim 1, characterized in that, Step 7 specifically includes: Calculate the imbalance based on the number of positive and negative samples: Where m + m is the number of positive samples. - This represents the number of negative samples. If d is less than d th If the sample is a minority class sample, then the number of data samples to be synthesized is calculated as follows: G = (m - -m + )×β, where β is used to control the balance level; for each sample x in the minority class i Calculate the ratio and normalize it: Where Δ i It is x i The number of samples belonging to the negative class among x's k nearest neighbors; calculate the number of samples belonging to the negative class among x's k nearest neighbors. i Number of samples to be generated: Thus, artificial positive sample x is obtained. i,j =x i +λ(x zi -x i ), where x zi It is x i The nearest neighbor, λ, is a random number; using this ADASYN method, m is generated that is similar to the features of the positive example. - -m + We use artificial positive samples to bring the dataset to an approximately balanced state and eliminate learning bias.

Images