A traffic planning course learning path tracking evaluation analysis method

Abstract

The present application belongs to the technical field of educational informatization, and relates to a traffic planning course learning path tracking evaluation analysis method, which obtains the flow link between the continuous links of investigation, expansion, calibration, prediction, etc. of the OD matrix, collects the operation records and AI tool calling records of the learners on the OD matrix in each link, extracts the data snapshot of the OD matrix in each link, then performs consistency check on the total data, row and column summary data and data distribution characteristics of the OD matrix of the adjacent links, marks the abnormal links introducing deviation according to the check result, traces back the deviation source link along the flow link in reverse, finally generates the evaluation result of the learners on the processing ability of the OD matrix, realizes the deviation source positioning and link-by-link quantitative evaluation, and distinguishes the learner's autonomous operation and the dependence on artificial intelligence tools, which is helpful to measure the mastery degree of the learners on the OD matrix processing flow of the traffic planning course.

Description

Technical Field

[0001] This invention belongs to the field of educational informatization technology and relates to a method for tracking, evaluating and analyzing learning paths in transportation planning courses. Background Technology

[0002] Traffic planning is a crucial course for transportation engineering majors. The origin-destination matrix (OD matrix), as a key data carrier describing the distribution of travel volume between traffic zones, is integrated into multiple teaching stages, including traffic surveys, data expansion, matrix calibration, and four-stage traffic demand forecasting. Learners' progressive processing and parameter transfer of the OD matrix at each stage are key indicators of their mastery of core traffic planning skills.

[0003] Currently, existing technologies for tracking and evaluating learning paths in transportation planning courses mainly employ the following methods: recording learners' resource click order, assignment submission time, video viewing nodes, and other temporal events through a course learning management system; statistically analyzing learners' learning behavior sequences based on this data; or constructing path clustering models based on learning behavior data to analyze the characteristics of the learning process path and identify different learning behavior patterns. In addition, some patented technologies focus on OD matrix estimation methods, calibration systems, or data processing devices. Their processing objects are usually actual OD data in the field of transportation engineering, aiming to improve the estimation accuracy or computational efficiency of the OD matrix, but do not involve tracking and evaluating the learners' OD matrix processing process in the teaching scenario.

[0004] The aforementioned existing technologies have at least the following drawbacks: 1. Existing learning path tracking methods take the learner's operation sequence as the tracking object, and can only record the specific execution steps completed. They cannot capture the data transfer and parameter change relationship between multiple continuous links such as investigation, expansion, calibration, and prediction of the OD matrix. They cannot determine whether the total amount data, row and column summary data, and data distribution characteristics of the OD matrix are consistent between adjacent links, nor can they reveal the data inconsistency problems that occur during data processing.

[0005] 2. When learners submit OD matrices with anomalies in subsequent stages, existing technologies can only determine that the stage results are unqualified. They cannot trace back the abnormality to the stage that caused it along the processing flow of the OD matrix, nor can they locate the specific source of the abnormality.

[0006] 3. Existing technologies can only evaluate the learner's final outcome and lack the ability to quantitatively evaluate the quality of the learner's operations at each stage of the OD matrix processing. Specifically, existing technologies struggle to identify whether learners used AI tools to complete the OD matrix processing at each stage, and whether they performed effective independent verification based on the AI ​​tool's output. Therefore, they cannot distinguish between operations performed autonomously by the learner and operations performed by AI tools, making it difficult to objectively evaluate the learner's true ability level. Summary of the Invention

[0007] In view of this, in order to solve the problems mentioned in the background technology, a learning path tracking evaluation and analysis method for transportation planning courses is proposed.

[0008] The objective of this invention can be achieved through the following technical solution: This invention provides a method for tracking and evaluating the learning path of a transportation planning course, comprising: obtaining the flow link of the OD matrix between multiple consecutive links in the transportation planning project processing flow; wherein the OD matrix is ​​a matrix that records the travel volume distribution data between each traffic zone;

[0009] Collect learners' operation records on the OD matrix in multiple consecutive stages, as well as records of the use of artificial intelligence tools in each stage, and extract data snapshots of the OD matrix in each stage;

[0010] Perform consistency checks on the data snapshots of the OD matrix between adjacent stages: compare the total data, row and column summary data, and data distribution characteristics in the OD matrix of adjacent stages to determine whether there are any discrepancies;

[0011] Based on the consistency verification results, the links that introduce deviations are marked as abnormal links, and the source of the deviations is traced back along the flow link.

[0012] Based on the source and anomaly stages, the learner's evaluation results on their OD matrix processing ability are generated.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention divides the OD matrix processing process into four stages: survey data acquisition and structuring, sample expansion, matrix calibration, and travel distribution prediction input, and establishes stage identifiers and flow order, and collects data snapshots of each stage. By comparing the total number of trips, row and column summary data and travel distance distribution sequence in the snapshots of adjacent stages, it is possible to directly determine whether the total OD matrix data, occurrence and attraction and distribution characteristics between adjacent stages are consistent, thus solving the problem that the prior art only records the operation sequence and cannot capture the relationship between OD matrix cross-stage data transmission and parameter changes.

[0014] (2) After detecting an anomaly in the OD matrix, this invention determines the range of deviation propagation based on the flow order of the source and the abnormal link in the data flow chain, and performs consistency verification and reverse tracing on adjacent links one by one along the data flow direction. In this way, it is possible to locate whether the deviation was introduced by the expansion link, the calibration link, or an earlier data acquisition link along the OD matrix processing flow, and determine the source link of the anomaly, overcoming the technical defect of the prior art that can only determine that the final result is unqualified but cannot trace the source of the anomaly in reverse.

[0015] (3) This invention collects operation records and AI tool call records at each stage, distinguishes between unmodified adoption and modified adoption based on the adoption method, and generates evaluation results by combining deviation type, deviation propagation range, and cause determination; at the same time, the consistency verification sub-items of each adjacent stage from the source stage to the abnormal stage are used to calculate the stage capability confidence level by proportion. Thus, it can quantitatively evaluate the learner's OD matrix processing quality at each stage, identify situations where learners rely on AI tool output in each stage without effective independent verification, and provide direct evidence for distinguishing between learner-driven operation and AI tool substitution. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the implementation steps of the method of the present invention;

[0018] Figure 2 This is a schematic diagram of the data transfer link and data snapshot acquisition process of the present invention;

[0019] Figure 3 This is a logical diagram illustrating the consistency verification of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention provides a learning path tracking and evaluation analysis method for transportation planning courses, specifically targeting the learning scenario in transportation planning courses where students use transportation planning software to complete the entire process of OD matrix survey data acquisition and structuring, sample expansion, calibration, and travel distribution prediction input.

[0022] In this embodiment, the OD matrix is ​​created and maintained by students in the traffic planning software. Rows in the OD matrix correspond to origin traffic zones, and columns correspond to destination traffic zones. Matrix elements record the travel volume between two traffic zones. Operation logs are collected through a listening plugin embedded in the traffic planning software. The listening plugin triggers callbacks when students perform write, modify, import, or export operations, recording the operation type, operation object identifier, operation timestamp, and differences in the OD matrix before and after the operation. Artificial intelligence tool call records are collected through a tool call listener. The listener monitors events where students call generative artificial intelligence tools to process the OD matrix, recording the input and output content of the tool call, and the learner's adoption of the output content. Data snapshots are generated by the data snapshot extraction unit at the end of each stage by reading the currently stored data content of the OD matrix.

[0023] In this invention, a traffic zone refers to a geographic statistical unit divided in traffic planning for analyzing the distribution of trip volume, and each traffic zone has a unique identifier; trip occurrence refers to the total number of trips originating from a traffic zone, which is represented in the OD matrix as the sum of the elements in the corresponding rows of the traffic zone; trip attraction refers to the total number of trips arriving at a traffic zone, which is represented in the OD matrix as the sum of the elements in the corresponding columns of the traffic zone.

[0024] Based on the above scenario, combined with Figure 1 As shown, the method of this embodiment includes the following steps: S1. Obtain the flow link of the OD matrix between multiple consecutive links in the traffic planning project processing flow; the OD matrix is ​​a matrix that records the travel volume distribution data between each traffic zone, wherein the matrix row index corresponds to the starting traffic zone, the column index corresponds to the ending traffic zone, and the matrix element represents the travel volume from the starting traffic zone to the ending traffic zone per unit time.

[0025] Reference Figure 2 As shown, one implementation of step S1 includes: S11. According to the traffic planning project processing flow, the processing sequence of the OD matrix is ​​divided into the OD survey data acquisition and structuring stage, the OD matrix sample expansion stage, the OD matrix calibration stage, and the travel distribution prediction input stage.

[0026] The OD survey data acquisition and structuring process involves receiving OD survey sample data entered or uploaded by users through their terminals, and aggregating the sample data according to origin traffic zones and destination traffic zones to generate an initial OD matrix. The sample data includes at least origin traffic zone identifiers, destination traffic zone identifiers, and trip volume.

[0027] The OD matrix sample expansion step is used to expand the initial OD matrix into a full OD matrix according to a preset expansion rule. As an optional implementation method, the preset expansion rule is: the reciprocal of the survey sampling rate is determined as the expansion coefficient, and each element in the initial OD matrix is ​​multiplied by the expansion coefficient to obtain the full OD matrix.

[0028] The OD matrix calibration step is used to align and calibrate the expanded full OD matrix with the current travel volume to obtain the calibrated full OD matrix.

[0029] The trip distribution prediction input stage uses the calibrated full OD matrix as input data for the trip distribution stage of the four-stage traffic demand prediction.

[0030] S12. Determine the data flow direction between each stage based on the input-output correspondence of OD matrix data between adjacent stages.

[0031] Specifically, an input OD matrix type and an output OD matrix type are defined for each stage; in adjacent stages, when the output OD matrix type of the previous stage matches the input OD matrix type of the next stage, a directed data flow relationship is established from the previous stage to the next stage.

[0032] The data flow direction determined as follows is: OD survey data acquisition and structuring, OD matrix sample expansion, OD matrix calibration, and travel distribution prediction input.

[0033] S13. Connect each link sequentially according to the data flow direction to form a flow link, and assign a link identifier and flow order to each link in the flow link.

[0034] The flow link is represented by a directed graph, where the nodes of the directed graph correspond to each processing stage, and the edges of the directed graph correspond to the data flow relationship between adjacent stages; the flow order of each stage is determined according to the topological order of the directed graph.

[0035] The link identifier is a globally unique identifier used to uniquely distinguish the corresponding link in the flow chain; the flow order is used to mark the sequential position of the corresponding link in the flow chain.

[0036] The link identifier and the flow order are associated and stored in the link description data, which together serve as the positioning benchmark for subsequent consistency verification and reverse tracing.

[0037] S2. Collect learners' operation records on the OD matrix in multiple consecutive stages, as well as records of calling artificial intelligence tools in each stage, and extract data snapshots of the OD matrix in each stage.

[0038] One implementation of step S2 includes: S21. Real-time monitoring of write, modify, import, and export operations performed by learners on the OD matrix at each stage through the event monitoring interface in the OD matrix processing software. In response to any monitored operation, the corresponding stage identifier, operation type, operation object identifier, operation timestamp, and differences in the OD matrix before and after the operation are obtained to form an operation record.

[0039] The operation object identifier includes the OD matrix file identifier, and when the operation targets a specific data element in the matrix, the starting traffic zone index and the ending traffic zone index corresponding to that specific data element.

[0040] The methods for obtaining pre-operation and post-operation data are as follows: In response to a detected write or modify operation, the full data of the current OD matrix is ​​obtained through the OD matrix data reading interface before the operation is executed, serving as the pre-operation data; after the operation is completed, the full data of the OD matrix is ​​obtained again through the same interface, serving as the post-operation data. The OD matrix data reading interface is adapted according to the type of traffic planning software used. For TransCAD software, it is read through its COM interface or API interface; for other traffic planning software, it is read through its corresponding data access interface or file parsing method.

[0041] S22. Set up a recording component at the call interface between the artificial intelligence tool and the OD matrix processing software to listen for events where learners call the artificial intelligence tool to process the OD matrix at each stage. In response to the monitored call events, record the stage identifier, input content, output content, and the learner's adoption method of the output content, forming an artificial intelligence tool call record.

[0042] The input includes the OD matrix data and / or prompts input by the learner into the AI ​​tool; the output includes the OD matrix data and / or processing suggestions returned by the AI ​​tool. Adoption is determined according to the following rules: The data after the AI ​​tool's output is actually written into the OD matrix is ​​obtained; if the actually written data matches the AI ​​tool's output and no subsequent modifications to the output are detected, it is considered an adoption without modification; if modifications are detected before or after the learner writes the AI ​​tool's output into the OD matrix, it is considered an adoption after modification.

[0043] The association between the output content of the AI ​​tool and subsequent modification operations is determined in the following way: the timestamp of the AI ​​tool's output content and the OD matrix object identifier involved in the output content are recorded; for each write or modification operation that occurs after the timestamp of the AI ​​tool's output content, if its operation object identifier is consistent with the object identifier of the AI ​​output content, and the difference between the operation timestamp and the timestamp of the AI ​​tool's output content is less than a preset time window (such as 300 seconds), then the write or modification operation is regarded as a subsequent modification operation on the AI ​​output content; if there is at least one modification operation, it is determined as a modified and adopted situation; otherwise, it is determined as an unmodified and adopted situation.

[0044] S23. In response to the end of the process event, read the data content currently stored in the OD matrix of the corresponding process, extract the total number of trips, the trip occurrence and trip attraction of each traffic zone, and the trip distribution structure as a data snapshot.

[0045] The end-of-stage event is triggered by detecting that the learner has performed an operation such as submitting, saving, or proceeding to the next stage on the current stage's processing result, to ensure that the data snapshot reflects the OD matrix data state after the corresponding stage processing is completed.

[0046] The total number of trips is the sum of all elements in the OD matrix; for any traffic zone... The number of trips is the OD matrix number. The sum of the elements of the line, the travel attraction is the first element of the OD matrix. The sum of the elements in the column.

[0047] The method for obtaining the travel volume distribution structure is as follows: according to the travel distance between each pair of origin traffic zones and destination traffic zones, the travel volume corresponding to each origin-destination pair in the OD matrix is ​​divided into a pre-set distance interval; the proportion of travel volume in each distance interval to the total travel volume is calculated, and the resulting proportion sequence is used as the travel volume distribution structure.

[0048] The data snapshot also includes a process identifier and a timestamp, and is stored in association with the corresponding process operation records and AI tool call records to support subsequent consistency verification and reverse tracing.

[0049] S3. Perform consistency verification on the data snapshots of the OD matrix between adjacent links: compare the total data, row and column summary data and data distribution characteristics in the OD matrix of adjacent links to determine whether there is any deviation.

[0050] Total data verification, row and column summary data verification, and data distribution characteristic verification, respectively, verify whether the OD matrix has been abnormally altered during the flow of data between adjacent stages from three dimensions: total balance, partition structure, and distance distribution. (Refer to...) Figure 3As shown, the specific implementation methods of the three are described in the following steps.

[0051] In this embodiment, comparing the total data in the OD matrix of adjacent links includes: for two adjacent links in the flow link, calculating the sum of the travel occurrences of each traffic zone in the data snapshot of the previous link and the data snapshot of the next link respectively, as the corresponding total occurrence, and the sum of the travel attraction of each traffic zone, as the corresponding total attraction.

[0052] The total number of trips generated and attracted are determined separately to satisfy the cross-stage total balance relationship. This cross-stage total balance relationship means that, in a normal OD matrix processing flow, steps such as sample expansion and calibration should not cause unfounded and significant changes to the total number of trips in the entire region. Therefore, under the constraint of a preset tolerance limit, the total number of trips generated and attracted between adjacent stages should remain continuous. The process of obtaining the preset tolerance limit is as follows: read the numerical precision of the OD matrix elements and determine the minimum unit of measurement. and obtain the number of traffic zones. , with the OD matrix in its entirety Based on the statistical distribution of the rounding errors of each element, when the rounding errors of each element are independently and identically distributed... When, the standard deviation of the total error is Based on the principle of 3 times the standard deviation of the normal distribution, a preset tolerance limit is taken as . This covers approximately 99.7% of normal error scenarios.

[0053] Specifically, when the absolute value of the difference between the total amount generated in the previous stage and the total amount generated in the next stage is less than or equal to the preset tolerance limit, it is determined that the total amount generated satisfies the cross-stage balance.

[0054] When the absolute value of the difference between the total amount attracted in the previous stage and the total amount attracted in the next stage is less than or equal to the preset tolerance limit, it is determined that the total amount attracted satisfies the cross-stage balance.

[0055] Compare the balance between the two stages. If at least one of the cross-stage balance relationships of total quantity or attraction quantity is not met, it is determined that there is a deviation in the total quantity data; and the type of indicator that does not meet the balance relationship, the direction of change, and the amount of deviation are recorded.

[0056] The deviation is the difference between the total amount of the next stage and the total amount of the previous stage. When the deviation is positive, it is recorded as a positive deviation of the total amount; when the deviation is negative, it is recorded as a negative deviation of the total amount.

[0057] It should be noted that for the same OD matrix, both the total amount generated and the total amount attracted are equal to the sum of all elements of the matrix, and they are identical within the same data snapshot. Therefore, the balance determination in this embodiment is aimed at the change of the same total index between adjacent links, rather than the equality relationship between the total amount generated and the total amount attracted within the same link.

[0058] In this embodiment, comparing the row and column summary data in the OD matrix of adjacent links includes: for two adjacent links in the flow link, calculating the trip generation and trip attraction of each traffic zone in the data snapshot of the previous link and the data snapshot of the next link respectively; the trip generation is the sum of the row elements of the matrix corresponding to the traffic zone, and the trip attraction is the sum of the column elements of the matrix corresponding to the traffic zone.

[0059] The proportion of trips generated in each traffic zone in the previous and subsequent stages to the total trips generated in all traffic zones is calculated to form the trip generation proportion distribution; and the proportion of trip attraction in each traffic zone to the total trip attraction in all traffic zones is calculated to form the attraction proportion distribution.

[0060] The application of proportional distribution is based on the fact that the processing of the OD matrix in the sample expansion and calibration stages is essentially a comprehensive scaling of the trip volume of each traffic zone. This comprehensive scaling does not change the relative proportional structure between the trip volumes of each traffic zone. Therefore, if the trip volume proportion of a certain traffic zone changes abnormally between adjacent stages, it indicates that the trip volume of that traffic zone has been rewritten individually, rather than undergoing reasonable comprehensive scaling.

[0061] The direction of change of the proportion of generation and the proportion of attraction between the two stages is determined by subtracting the proportion value of the previous stage from the proportion value of the later stage: when the difference is positive, the proportion increases; when the difference is negative, the proportion decreases; when the difference is zero or less than the minimum resolution determined by the data storage accuracy, the proportion remains unchanged.

[0062] Based on the comparison of travel volume in each traffic zone from the previous data snapshot, the traffic zone with the highest travel volume is identified. Using the traffic zone with the highest travel volume as the reference zone, traffic zones that meet one of the following conditions are marked as deviation zones: a. The direction of change is opposite to the direction of change of the reference zone; the basis for applying condition a is: the traffic zone with the highest travel volume has a large base of travel volume and stable data, and under the overall scaling or unified calibration operation, its direction of change is consistent with the overall operation direction; the zone with the opposite direction of change indicates that it has been adjusted separately in a way that contradicts the overall operation.

[0063] b. After sorting the occurrence ratio or attraction ratio of the previous and subsequent stages in descending order of value, the ranking changes; the basis for applying condition b is: the overall scaling does not change the relative order of travel volume between traffic zones. If the ranking of a certain traffic zone changes between adjacent stages, it means that the travel volume of that traffic zone has been rewritten separately, which has destroyed the relative order of the overall scaling.

[0064] c. The direction of change in the proportion of production is opposite to the direction of change in the proportion of attraction.

[0065] If there are marked off-zones, it is determined that there is a deviation in the row and column summary data, and the corresponding traffic zone identifier is recorded.

[0066] In this embodiment, comparing the data distribution characteristics in the OD matrix of adjacent links includes: for two adjacent links in the flow link, merging the travel volume of traffic partition pairs in the OD matrix according to several equally spaced distance intervals, to obtain the travel distance distribution sequence corresponding to the previous link and the next link respectively. ,in This represents the total number of travel distance intervals.

[0067] Determine whether the travel distance distribution sequence of the previous and subsequent stages satisfies the decay trend of decreasing travel volume as the travel distance increases. The decay trend refers to the distribution law that the travel volume decreases as the travel distance increases. Its application is based on the fact that traffic travel follows the distance decay law, which is specifically manifested in that the travel volume between traffic zones decreases as the travel distance (impedance) between the two places increases, and the proportion of short-distance travel volume is greater than the proportion of long-distance travel volume.

[0068] When determining the attenuation trend, the distance intervals are arranged from nearest to farthest. The proportion of travel volume in adjacent distance intervals is compared sequentially. For any adjacent relationship, if the proportion of the subsequent distance interval does not exceed the sum of the proportion of the preceding distance interval and the relaxation tolerance, the adjacent relationship is determined to satisfy the attenuation trend; otherwise, it is not satisfied. If all adjacent relationships satisfy the attenuation trend, the travel distance distribution sequence of the corresponding stage is determined to satisfy the attenuation trend; otherwise, it is not satisfied.

[0069] Among them, relaxation tolerance The absolute percentage difference tolerance is used to accommodate statistical fluctuations that may exist in actual traffic survey data. The determination process is as follows: Obtain the travel distance distribution sequence from the previous stage, calculate the absolute value of the difference in travel volume percentage between adjacent distance intervals, and take the median as the baseline fluctuation value. Multiply the baseline fluctuation value by a preset multiple to obtain the relaxation tolerance. When there is no data from the previous stage, take the median of the absolute values ​​of the difference in travel volume percentage between adjacent distance intervals in the current stage's travel distance distribution sequence as the baseline fluctuation value, and then multiply it by the preset multiple to obtain the baseline fluctuation value. The reason for using the median as a benchmark fluctuation value is that the median is robust to extreme values. In the actual distribution of travel distances, individual adjacent distance intervals may experience local fluctuations due to urban spatial structure (such as a polycentric structure). The median can resist the interference of such local outliers on the estimation of the benchmark fluctuation level.

[0070] The preferred value for the preset multiplier is 2, which means that when the proportion of travel volume in the subsequent interval exceeds twice the normal adjacent fluctuation range of the preceding interval, it is determined that the reverse change has exceeded the normal fluctuation range and constitutes reverse attenuation. The multiplier can be preset and adjusted according to the determination sensitivity requirements.

[0071] Comparing the decay trends of the preceding and following stages: Inverse decay segment detection: For the travel distance distribution sequence of the following stage, compare the values ​​of the preceding and following stages sequentially. The and the first The percentage of trips within each distance range ( ),when At that time, the first Each distance interval is marked as an inverse attenuation segment.

[0072] Attenuation trend deviation detection: Calculate the difference in the proportion of each distance interval between the previous and subsequent stages, and use the absolute maximum value of the corresponding difference in the historical operation records as the deviation threshold. When there are no historical operation records, the relaxation tolerance is directly used as the deviation threshold. If any distance interval satisfies the condition that the proportion difference is greater than the deviation threshold, it is determined that the distance interval has deviated significantly. When the number of distance intervals with significant deviation is not less than one, it is determined that the attenuation trend of the two stages has deviated.

[0073] When a reverse attenuation segment is detected, or when the attenuation trend is determined to be deviated, the data distribution characteristics are determined to be biased. The type of bias, the corresponding distance interval number and range, the proportion before and after, the difference in proportion, the associated link identifier and snapshot identifier are recorded and stored for subsequent consistency verification and reverse tracing.

[0074] S4. Based on the consistency verification results, mark the links that introduce deviations as abnormal links, and trace the source of the deviations back along the flow link.

[0075] One implementation of step S4 includes: after performing consistency verification on each pair of adjacent links in the flow link, determining the adjacent link pairs where at least one of the total data, row and column summary data, and data distribution characteristics is determined to have deviation as deviation link pairs.

[0076] For each deviation pair, the pair that introduced the deviation is determined based on the type of deviation: when the balance of the total data changes from satisfied to unsatisfied, the deviation is determined to be introduced by the next pair of deviation pairs; when the balance changes from unsatisfied to satisfied, the deviation is determined to be introduced by the previous pair of deviation pairs. The criteria for determination are as follows: when the balance changes from satisfied to unsatisfied, it indicates that the processing in the next pair disrupts the balance of the total data, and the deviation is introduced by the next pair; when the balance changes from unsatisfied to satisfied, it indicates that the balance deficiency left by the previous pair has been corrected by the next pair, and the deviation is introduced by the previous pair.

[0077] The change from a satisfied to a dissatisfied balance state refers to the following: the total balance check between the preceding stage and its upstream neighboring stage passes, while the total balance check between the following stage and the preceding stage fails, in which case the deviation is introduced by the following stage. Conversely, the change from a dissatisfied to a satisfied balance state refers to the following: the total balance check between the preceding stage and its upstream neighboring stage fails, while the total balance check between the following stage and the preceding stage passes, in which case the deviation originates from the preceding stage, and the following stage corrects the balance deficiency left by the preceding stage.

[0078] When the deviation type is row and column summary deviation or distribution characteristic deviation, the deviation is determined to be introduced by the next step in the deviation process. This is because row and column summary deviation and distribution characteristic deviation detect unreasonable data changes in the next step relative to the previous step, which are introduced by the processing operation of the next step.

[0079] The steps that are determined to introduce deviations are marked as abnormal steps.

[0080] Record the deviation types corresponding to abnormal processes; the deviation types include total deviation, row and column summary deviation, and distribution characteristic deviation.

[0081] Starting from the abnormal link, backtrack step by step in the opposite direction of the data flow in the data flow link;

[0082] Perform a consistency check between the data snapshot of each stage accessed during the rollback process and the data snapshot of its upstream adjacent stage;

[0083] The downstream adjacent links of the links that show no deviation during the first verification in the rollback process are identified as the source links of the deviation.

[0084] When rolling back to the first link in the flow chain, if all the checks that have been performed have been found to be biased, then the link where the last check in the rollback process was found to be biased is determined as the source link of the bias.

[0085] S5. Based on the source and anomaly stages, generate the learner's evaluation results on the OD matrix processing ability.

[0086] One implementation of step S5 includes: based on the flow order of the source link and the abnormal link in the flow chain, taking the source link as the starting point and the abnormal link as the ending point, obtaining all intermediate links between the two along the data flow direction of the OD matrix, and forming an ordered set of links together with the source link, intermediate links and abnormal links as the deviation propagation range; when the source link and the abnormal link are the same, the deviation propagation range only includes the source / abnormal link.

[0087] The operation records and AI tool call records corresponding to the source link are read to determine the cause of the deviation. The cause of the deviation is introduced by the learner's operation, introduced by the AI ​​tool output, or other reasons. The specific process is as follows: In the AI ​​tool call records of the source link, the call events with the adoption method of "adopted without modification" are searched. When the corresponding event is found, it is determined that the deviation is related to the AI ​​tool output, and an AI tool dependency prompt is added to the evaluation result.

[0088] In the source operation records, operations of the type "write," "modify," "import," or "export," whose operation object identifier matches the OD matrix object involved in the current deviation, and whose operation time is earlier than the time when the deviation first appeared, constitute a candidate operation set. For each candidate operation, read the pre-operation and post-operation data, and perform consistency checks on the total data, row and column summary data, and data distribution characteristics: when there is no deviation in the pre-operation data but a deviation exists in the post-operation data, it is determined that the deviation was introduced by the learner's operation, the operation type and operation time are recorded, and a prompt indicating insufficient operational ability is added to the evaluation results;

[0089] If the data before the operation already has a deviation and the data after the operation still has a deviation, it is judged as an invalid modification. At this time, if there is an event in the source link's artificial intelligence tool call record that is adopted after modification and is related to the invalid modification operation in terms of object and time, then add an artificial intelligence dependency and no effective verification prompt to the evaluation result; otherwise, add an invalid operation correction prompt.

[0090] Among them, the correlation in terms of object and time refers to the fact that the event record in the AI ​​tool call record meets all of the following conditions: the operation object identifier in the event record is the same as the object identifier of the invalid modification operation; the adoption method field in the event record is "adopted after modification"; the absolute value of the difference between the timestamp of the event record and the occurrence time of the invalid modification operation is less than or equal to the time correlation threshold, which is determined as follows: take the median and interquartile range of the absolute values ​​of the time differences of all modified adoption events in the same source link, and use the sum of the median and 1.5 times the interquartile range as the time correlation threshold.

[0091] If the data before the operation has a deviation but the data after the operation has no deviation, it is determined that the operation has effectively corrected the deviation, and no insufficient operation capability prompt is added.

[0092] If there is no deviation between the data before and after the operation, the operation is determined to be unrelated to the current deviation, and the next candidate operation is checked.

[0093] When the candidate operation set is empty and there are no related events in the AI ​​tool call record, it is determined that the specific cause cannot be located, and other cause indicators are generated.

[0094] By combining the type of deviation, the scope of deviation propagation, and the source of deviation, an evaluation result on the learner's ability to process the OD matrix is ​​generated; the evaluation result includes: deviation location information including the source link identifier, the abnormal link identifier, and the link identifier sequence within the scope of deviation propagation;

[0095] Indications of bias causes include at least one of the following: introduction of artificial intelligence tool output, introduction of learner operation, or others;

[0096] The capability deficiency prompt includes at least one of the following: AI tool dependency prompt, AI dependency without effective verification prompt, insufficient operational capability prompt, and ineffective operation correction prompt.

[0097] The link capability confidence is determined by the consistency verification results of each adjacent link pair between the source link and the abnormal link. Specifically, for each adjacent link pair, the proportion of the number of successful consistency verification items to the total number of items is taken as the confidence of the adjacent link pair; the overall link confidence is the minimum value of the confidence of each adjacent link pair.

[0098] The evaluation results are output in structured data format and can be stored in association with learner identifiers, OD matrix processing project identifiers, and corresponding data snapshot identifiers. When there are no records of AI tool calls and operations at the source stage, it is determined that the specific cause cannot be located, and a corresponding prompt is generated.

[0099] The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, and all such modifications or additions should fall within the protection scope of the present invention.

Claims

1. A method for tracking and evaluating learning paths in a transportation planning course, characterized in that, include: Obtain the flow links of the OD matrix between multiple consecutive stages in the traffic planning project processing flow; The OD matrix is ​​a matrix that records the distribution data of travel volume between each traffic zone; Collect learners' operation records on the OD matrix in multiple consecutive stages, as well as records of the use of artificial intelligence tools in each stage, and extract data snapshots of the OD matrix in each stage; Perform consistency checks on the data snapshots of the OD matrix between adjacent stages: compare the total data, row and column summary data, and data distribution characteristics in the OD matrix of adjacent stages to determine whether there are any discrepancies; Based on the consistency verification results, the links that introduce deviations are marked as abnormal links, and the source of the deviations is traced back along the flow link. Based on the source and anomaly stages, the learner's evaluation results on their OD matrix processing ability are generated.

2. The method for tracking and evaluating learning paths in a transportation planning course according to claim 1, characterized in that, The flow link of the OD matrix in the traffic planning project processing flow between multiple consecutive stages includes: According to the traffic planning project processing flow, the processing sequence of the OD matrix is ​​divided into the OD survey data acquisition and structuring stage, the OD matrix sample expansion stage, the OD matrix calibration stage, and the travel distribution prediction input stage. Based on the input-output correspondence of OD matrix data between adjacent stages, the direction of data flow between each stage is determined; The links are connected sequentially according to the direction of data flow to form a flow chain, and each link in the flow chain is assigned a link identifier and flow order.

3. The method for tracking and evaluating learning paths in a transportation planning course according to claim 1, characterized in that, The extraction of data snapshots of the OD matrix at each stage includes: Monitor the learners' write, modify, import, and export operations on the OD matrix at each stage, record the operation type, operation object identifier, operation timestamp, and the differences in the OD matrix before and after the operation, and form an operation record; Monitor events in each stage where learners call artificial intelligence tools to process the OD matrix, record the input content, output content, and how learners adopt the output content, and form an artificial intelligence tool call record; In response to the end of a stage event, the data content currently stored in the corresponding stage's OD matrix is ​​read, and the total number of trips, the trip occurrence and attraction of each traffic zone, and the trip distribution structure are extracted as a data snapshot.

4. The method for tracking and evaluating learning paths in a transportation planning course according to claim 3, characterized in that, Compare the total data in the OD matrix of adjacent stages, including: For two adjacent links in the flow link, calculate the sum of the travel occurrences of each traffic zone in the data snapshot of the previous link and the data snapshot of the next link, as the corresponding total occurrence, and the sum of the travel attraction of each traffic zone, as the corresponding total attraction. Determine whether the total amount generated and the total amount attracted satisfy the cross-stage total balance relationship; By comparing the balance between the previous and subsequent stages, if at least one of the cross-stage balance relationships of total quantity or cross-stage balance relationship of attracting total quantity is not met, it is determined that there is a deviation in the total quantity data; and the type of indicator that does not meet the balance relationship, the direction of change, and the amount of deviation are recorded.

5. The method for tracking and evaluating learning paths in a transportation planning course according to claim 3, characterized in that, Compare the row and column summaries in the OD matrix of adjacent stages, including: For two adjacent links in the flow link, calculate the trip generation and trip attraction of each traffic zone in the data snapshot of the previous link and the data snapshot of the next link respectively; the trip generation is the sum of the row elements of the matrix corresponding to the traffic zone, and the trip attraction is the sum of the column elements of the matrix corresponding to the traffic zone. Calculate the proportion of trip occurrences in each traffic zone to the total trip occurrences in all traffic zones in the previous and subsequent stages to form the occurrence proportion distribution; and calculate the proportion of trip attraction in each traffic zone to the total trip attraction in all traffic zones to form the attraction proportion distribution. The direction of change of the proportion of generation and the proportion of attraction between the two stages is determined for each zone, and the direction of change includes increasing proportion, decreasing proportion and remaining unchanged proportion; Using the traffic zone with the highest traffic volume as the reference zone, traffic zones that meet one of the following conditions are marked as deviating from the zone: a. The direction of change is opposite to the direction of change of the reference partition; b. After sorting the production or attraction ratios of the previous and subsequent stages from high to low according to their values, the ranking changes. c. The direction of change in the proportion of output is opposite to the direction of change in the proportion of absorption. If there are marked off-zones, it is determined that there is a deviation in the row and column summary data, and the corresponding traffic zone identifier is recorded.

6. The method for tracking and evaluating learning paths in a transportation planning course according to claim 3, characterized in that, Compare the data distribution characteristics in the OD matrix of adjacent stages, including: For two adjacent links in the flow link, the travel volume in the OD matrix is ​​divided into multiple travel distance intervals according to the travel distance between the traffic zone pairs, so as to obtain the travel distance distribution sequence corresponding to the previous link and the next link respectively. Determine whether the travel distance distribution sequences of the preceding and following stages satisfy the decay trend of decreasing travel volume as the travel distance increases; Compare the decay trends of the previous and subsequent stages: When a reverse decay segment appears in the travel distance distribution sequence of the subsequent stage that is inconsistent with the trend of the previous stage, or when the decay trends of the two stages deviate, it is determined that there is a deviation in the data distribution characteristics, and the corresponding travel distance interval is recorded.

7. The method for tracking and evaluating learning paths in a transportation planning course according to claim 1, characterized in that, The step of marking the process that introduces deviation as an abnormal step includes: After performing consistency verification on each pair of adjacent links in the flow link, the adjacent link pairs in which at least one of the total data, row and column summary data and data distribution characteristics is determined to have deviation are identified as deviation link pairs. For each deviation pair, determine the pair that introduced the deviation based on the type of deviation: when the balance of the total data changes from satisfied to unsatisfied, the deviation is determined to be introduced by the next pair of deviations; when the balance changes from unsatisfied to satisfied, the deviation is determined to be introduced by the previous pair of deviations; when the type of deviation is row / column summary deviation or distribution characteristic deviation, the deviation is determined to be introduced by the next pair of deviations. The steps identified as introducing deviations are marked as abnormal steps; Record the deviation types corresponding to abnormal processes; the deviation types include total deviation, row and column summary deviation, and distribution characteristic deviation.

8. The method for tracking and evaluating learning paths in a transportation planning course according to claim 1, characterized in that, The sources of the deviation traced in reverse along the transfer link include: Starting from the abnormal link, backtrack step by step in the opposite direction of the data flow in the data flow link; Perform a consistency check between the data snapshot of each stage accessed during the rollback process and the data snapshot of its upstream adjacent stage; The downstream adjacent links of the links that show no deviation during the first verification in the rollback process are identified as the source links of the deviation. When rolling back to the first link in the flow chain, if all adjacent link checks performed during the rollback from the abnormal link to the first link show deviations, then the downstream adjacent link in the last adjacent link check performed during the rollback process is identified as the source of the deviation.

9. The method for tracking and evaluating learning paths in a transportation planning course according to claim 1, characterized in that, The evaluation results of the learners' ability to process the OD matrix include: Based on the flow order of the source link and the abnormal link in the circulation link, determine the set of links through which the deviation propagates from the source link to the abnormal link, as the deviation propagation range; Read the operation records and AI tool call records corresponding to the source link to determine the cause of the deviation. The cause of the deviation is one of the following: introduced by the learner's operation, introduced by the AI ​​tool output, or other. By combining the type of deviation, the scope of deviation propagation, and the source of deviation, an evaluation result of the learner's ability to process the OD matrix is ​​generated. The evaluation result includes deviation location information, deviation cause indication, capability deficiency prompts, and the confidence level of the link capability determined by the consistency verification results of each adjacent link from the source link to the abnormal link.

10. The method for tracking and evaluating learning paths in a transportation planning course according to claim 9, characterized in that, The causes of the determination deviation include: Search for unmodified and adopted call events in the AI ​​tool call records at the source stage. If found, determine that the deviation was introduced by the AI ​​tool output and generate an AI tool dependency prompt. In the operation records of the source stage, operations of the type of writing, modification, import or export, operation objects that are consistent with the OD matrix involved in the current deviation, and operation time that is earlier than the time when the deviation first appeared are selected to form a candidate operation set; Read the data before and after the candidate operation, and perform a consistency check: If the data is without deviation before the operation but has deviation after the operation, the deviation is determined to be caused by the learner's operation, and a prompt indicating insufficient operation ability is generated. If the data before the operation is biased and the data after the operation is still biased, it is determined to be an invalid modification; if the invalid modification operation is associated with the AI ​​tool call event with the adoption method of modification and adoption in terms of object and time, an AI dependency is generated and there is no effective verification prompt; otherwise, an invalid operation correction prompt is generated. If the candidate operation set is empty and there are no associated AI tool call events, then generate alternative cause indicators.

Images