Train human-computer interface display intelligent analysis method based on artificial intelligence

Abstract

The invention relates to the technical field of rail transit train intelligent monitoring, and discloses a train human-computer interface display intelligent analysis method based on artificial intelligence. According to the method, the stability of a display element is evaluated and a potential fault path is predicted by analyzing a dynamic evolution track of an interface state in a historical period. And applying simulation operation to the logic conflict nodes, separating the underlying service features and the user operation behavior features from the mixed response, and respectively carrying out independent traceability positioning. According to the method, early warning of abnormal behaviors of interface elements can be realized, different levels of faults caused by system service defects and interaction logic errors are accurately distinguished and positioned, and a core technical means is provided for intelligent monitoring and efficient maintenance of a train human-computer interface.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring technology for rail transit trains, specifically to an intelligent analysis method for train human-machine interface display based on artificial intelligence. Background Technology

[0002] Existing technologies for monitoring and diagnosing the display status of train human-machine interfaces (HMIs) primarily rely on static compliance checks based on preset rules and threshold judgments of response time. Static compliance checks compare the current interface image with a standard template to determine whether the displayed content is correct and complete; this method can only capture static, explicit errors. Response time monitoring focuses on the overall time from the issuance of an operation command to the completion of the interface update; any timeout is considered abnormal. These two methods are currently the mainstream means of ensuring the usability of train HMIs.

[0003] The aforementioned existing technical solutions have shortcomings. Static inspection cannot handle logical conflicts related to time or operation sequences that arise during dynamic interactions. While response time monitoring can detect performance degradation, it cannot distinguish whether the root cause of the anomaly lies in the latency of the underlying data service, the failure of the interface rendering engine, or the inefficient processing logic triggered by a specific combination of operations. After a failure occurs, operations and maintenance personnel often need to rely on experience to perform a large amount of manual testing and log analysis, which is inefficient in locating problems and easily overlooks deep-seated issues.

[0004] This invention aims to address the problems of predicting potential faults from dynamic interactions and accurately separating and locating root causes at different levels when anomalies occur. This requires the technical solution to understand the long-term behavioral patterns of interface elements and to have the ability to decouple mixed response signals at their source. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent analysis method for train human-machine interface display based on artificial intelligence, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides an intelligent analysis method for train human-machine interface display based on artificial intelligence, the method comprising: The system acquires the interface image stream of the target human-machine interface from the train operation monitoring system, outputs interface description information containing semantic tags, and obtains the verified real-time interface status. Extract the evolution trajectory of the real-time interface state in multiple historical periods, identify the pattern deviation interval in the evolution trajectory, and calculate the stability quantification value of the interface display elements based on the pattern deviation interval. The stability quantification value is input into the fault knowledge graph to match potential fault evolution paths. Based on the potential fault evolution paths, the logical conflict nodes in the target human-machine interface are deduced in reverse. A simulated operation sequence is applied to the logical conflict node to capture the response features of the target human-machine interface, and service features generated by the underlying display service and behavioral features generated by user operation behavior are separated from the response features. The service features and the behavior features are traced independently to locate the service anomaly source and the behavior anomaly source. Multi-dimensional analysis tasks are constructed, and parallel diagnostic schemes are arranged sequentially according to the security analysis procedure. An analysis instruction list is generated, loaded into the simulation test environment to drive the full process simulation, and the final output results are collected. The final output result is evaluated for compliance, and combined with the multi-dimensional analysis task, a complete intelligent analysis conclusion for the target human-computer interface is formed.

[0007] Preferably, the step of acquiring the interface image stream of the target human-machine interface from the train operation monitoring system, outputting interface description information containing semantic tags, and obtaining the verified real-time interface state includes: Identify the structured display elements in the interface image stream, compare the structured display elements with entries in a preset interface knowledge base, and output interface description information containing semantic tags; Dynamic intent mapping is performed on the interface description information to generate an interface state hypothesis under the current operation scenario, and the interface state hypothesis is verified by calling historical logs to obtain the verified real-time interface state. The step of dynamically mapping the interface description information to generate interface state assumptions for the current operation scenario includes: Parse the semantic tags in the interface description information and identify the functional component types corresponding to the semantic tags; Obtain the current train's operating mode parameters, associate and match the operating mode parameters with the functional component types, and determine the current operating scenario; Based on the current operation scenario, a preset scenario-state mapping rule is queried to generate an interface state hypothesis corresponding to the interface description information; Iterate through the assumptions contained in the interface state assumptions and check the feasibility of each assumption in the target human-computer interface. Hypothetical conditions whose feasibility exceeds a preset threshold are selected, and based on the hypothetical conditions, interface state hypotheses for the current operation scenario are generated.

[0008] Preferably, the step of calling historical logs to verify the interface state hypothesis and obtain the verified real-time interface state includes: Retrieve the historical operation logs associated with the target human-machine interface from the train's central data server; Search the historical operation log for record segments that are temporally adjacent to the assumed interface state. Extract the interface response data recorded in the recorded segment; The interface response data is compared one by one with the expected data of the interface state assumption, and the consistent items and inconsistent items are recorded. The total number of the consistent items is counted. When the total number reaches the preset verification pass standard, the interface state assumption is determined to be valid. The interface state assumption is fused with the latest time data in the interface response data to form the verified real-time interface state.

[0009] Preferably, the step of extracting the evolution trajectory of the real-time interface state over multiple historical periods and identifying the pattern deviation interval in the evolution trajectory includes: Set a fixed time window length, and extract multiple consecutive historical period data from the historical database corresponding to the real-time interface status based on the time window length; Extract the key attribute values ​​of the real-time interface state from each of the historical period data; The key attribute values ​​are arranged in chronological order to form the evolution trajectory of the real-time interface state in multiple historical periods. A baseline behavior pattern curve is established on the evolution trajectory; Calculate the difference between the data at each time point on the evolution trajectory and the corresponding point on the baseline behavior pattern curve, and mark the continuous time area where the difference exceeds the preset fluctuation range as the pattern deviation interval.

[0010] Preferably, the step of calculating the stability quantification value of the interface display element based on the mode deviation interval includes: The cumulative duration and frequency of the deviation intervals of the described pattern within the total analysis period are statistically analyzed. Measure the magnitude of the numerical change of the displayed element within each deviation interval of the stated pattern; Obtain the standard variation range of the interface display elements in normal working mode; Based on the cumulative duration, the frequency of occurrence, the magnitude of numerical change, and the standard range of change, a stability evaluation function is constructed. The cumulative duration, the frequency of occurrence, and the magnitude of numerical change are input into the stability evaluation function to calculate the stability quantification value of the interface display element.

[0011] Preferably, the step of inputting the stability quantification value into the fault knowledge graph to match potential fault evolution paths includes: Construct a fault knowledge graph with interface display elements as nodes and logical dependencies and signal transmission relationships between elements as edges; In the fault knowledge graph, each node is labeled with its historical stability quantification value range and corresponding typical fault mode; The calculated stability quantification value of the interface display element is compared with the historical stability quantification value range of the corresponding node in the fault knowledge graph. If the stability quantification value falls within the range of historical stability quantification values ​​corresponding to a typical failure mode, then the path associated with the typical failure mode in the failure knowledge graph is activated. From the activated paths, paths that meet the current train operation constraints are selected as the potential fault evolution paths.

[0012] Preferably, the step of reversely deriving the logical conflict nodes in the target human-machine interface based on the potential fault evolution path includes: Starting from the terminal fault phenomenon of the potential fault evolution path, trace back in reverse along the edges in the fault knowledge graph; During the reverse tracing process, all nodes visited and the causal relationships between nodes are recorded; Analyze the causal relationships to identify causal chains that contain contradictions or whose conditions are not met; Locate the node that the contradictory or unmet causal chains all point to; The node is identified as a logical conflict node in the target human-machine interface.

[0013] Preferably, the step of applying a simulated operation sequence to the logical conflict node and capturing the response characteristics of the target human-machine interface includes: Based on the functional definition of the logical conflict nodes, design a simulated operation sequence that covers normal operation, boundary operation, and abnormal operation; In a simulation environment, the simulated operation sequence is used as input and applied to the target human-machine interface containing the logical conflict node; Real-time monitoring of the target human-machine interface's internal state variables and external output signals during the processing of the simulated operation sequence; Record the change process of the internal state variables and the timing waveform of the external output signal; Feature data that characterizes the dynamic behavior of the target human-machine interface is extracted from the recorded change process and the time-series waveform, and used as the response features.

[0014] Preferably, the steps involve independently tracing the service characteristics and behavioral characteristics to locate the service anomaly source and the behavioral anomaly source, constructing a multi-dimensional analysis task, sequentially arranging the parallel diagnostic scheme according to the security analysis procedure, generating an analysis instruction list, loading it into the simulation test environment to drive the full-process simulation, and collecting the final output results, including: Locate the service anomaly source corresponding to the service feature, and the behavior anomaly source corresponding to the behavior feature; Based on the service anomaly source and the behavior anomaly source, a multi-dimensional analysis task is constructed, which includes a parallel diagnostic scheme for the target human-machine interface. Based on the preset security analysis procedure, the parallel diagnostic scheme is sequentially arranged to generate an executable analysis instruction list; The analysis instruction list is sequentially loaded into the simulation test environment where the target human-machine interface is located, driving the target human-machine interface to perform a full-process simulation. After the full-process simulation is completed, the final output result of the target human-machine interface is collected.

[0015] Preferably, the step of performing a conformity determination on the final output result and combining it with the multi-dimensional analysis task to form a complete intelligent analysis conclusion for the target human-computer interface includes: The final output results are used to parse the interface display content, response time, and data accuracy indicators. The interface display content, response time, and data accuracy indicators are compared one by one with the requirements in the predefined system specifications. Based on the comparison results, a judgment conclusion is given for each indicator item: it meets the requirements, it does not meet the requirements, or there is a deviation. Summarize the judgment conclusions of all indicator items, and combine them with the service anomaly source and behavior anomaly source information found during the execution of the multi-dimensional analysis task; Based on the combined judgment conclusions and the information on the service anomaly sources and behavior anomalies, a structured diagnostic report is generated. This structured diagnostic report represents the complete intelligent analysis conclusions of the target human-machine interface.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This system extracts the evolution trajectory of the interface state over multiple historical periods, identifies deviations in patterns, and calculates a quantitative value for the stability of the interface elements. Expanding the analysis from static snapshots to dynamic time series allows for the quantification of the predictability and consistency of each interface element's behavior. This enables the system to identify interface elements that, while not violating static rules, exhibit subtle deviations in their behavior patterns and show unstable trends, achieving early warning and trend assessment of potential faults. Fault path matching based on quantified stability values ​​shifts its prediction basis from a single rule to a comprehensive evaluation combining historical behavior patterns, enhancing the targetedness and foresight of the warnings.

[0017] By separating service characteristics generated by the underlying display service and behavioral characteristics generated by user operation logic from the overall interface response characteristics triggered by simulated operations, and independently tracing the source of each to pinpoint the service anomaly and behavioral anomaly source respectively, this approach achieves decoupled analysis of complex mixed signals, breaking the traditional limitation of treating the interface response as a complete black box for diagnosis. Through separation and tracing, it is possible to clearly distinguish whether an interface anomaly originates from a problem in the backend data service, communication link, or rendering engine, or from a defect in the frontend interface logic's handling of a specific operation sequence. Precise source location allows maintenance measures to directly address the core problem, avoiding blind troubleshooting across different layers of the system and improving diagnostic efficiency and repair accuracy. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the working principle of the intelligent analysis method for train human-machine interface display based on artificial intelligence as described in this invention. Figure 2 A flowchart for verifying the interface state assumptions; Figure 3 A flowchart for identifying pattern deviation intervals; Figure 4 A confidence analysis diagram of logical conflict nodes in the train's human-machine interface. Figure 5 A comparison chart showing the stability scores of elements in the train's human-machine interface. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1This invention provides an intelligent analysis method for train human-machine interface displays based on artificial intelligence. The method includes: acquiring an interface image stream of the target human-machine interface from a train operation monitoring system, outputting interface description information containing semantic tags, and obtaining a verified real-time interface state; extracting the evolution trajectory of the real-time interface state over multiple historical periods, identifying pattern deviation intervals in the evolution trajectory, and calculating stability quantification values ​​of interface display elements based on the pattern deviation intervals; inputting the stability quantification values ​​into a fault knowledge graph, matching potential fault evolution paths, and, based on the potential fault evolution paths, reverse-engineering logical conflict nodes in the target human-machine interface; and further analyzing the data. The logical conflict node applies a simulated operation sequence to capture the response features of the target human-machine interface. From these response features, service features generated by the underlying display service and behavioral features generated by user operations are separated. The service features and behavioral features are independently traced to pinpoint the service anomaly source and the behavioral anomaly source, respectively. A multi-dimensional analysis task is constructed, and parallel diagnostic schemes are sequentially arranged according to security analysis procedures to generate an analysis instruction list. This list is loaded into the simulation test environment to drive the entire process simulation, and the final output results are collected. The final output results are then evaluated for compliance. Combined with the multi-dimensional analysis task, a complete intelligent analysis conclusion for the target human-machine interface is formed.

[0021] In one embodiment of the present invention, see [reference] Figure 2The system identifies structured display elements in the interface image stream, compares these elements with entries in a preset interface knowledge base, and outputs interface description information containing semantic tags. It then performs dynamic intent mapping on the interface description information to generate interface state hypotheses for the current operating scenario, and verifies these hypotheses using historical logs to obtain a verified real-time interface state. The specific process of dynamically mapping the interface description information to generate interface state hypotheses for the current operating scenario includes: parsing the semantic tags in the interface description information and identifying the functional component types corresponding to the semantic tags; obtaining the current train's operating mode parameters and associating and matching these parameters with the functional component types to determine the current operating scenario; querying preset scenario-state mapping rules based on the current operating scenario to generate interface state hypotheses corresponding to the interface description information; traversing the hypotheses contained in the interface state hypotheses and verifying their feasibility in the target human-machine interface; filtering out hypotheses whose feasibility exceeds a preset threshold, and generating interface state hypotheses for the current operating scenario based on these hypotheses. The specific process of verifying the interface state hypothesis by calling historical logs to obtain the verified real-time interface state includes: retrieving historical operation logs associated with the target human-machine interface from the train's central data server; searching for record segments in the historical operation logs that are temporally adjacent to the interface state hypothesis; extracting the interface response data recorded in the record segments; comparing the interface response data with the expected data of the interface state hypothesis one by one, recording consistent and inconsistent items; counting the total number of consistent items, and when the total number reaches a preset verification pass standard, determining that the interface state hypothesis is valid; and merging the interface state hypothesis with the latest time point data in the interface response data to form the verified real-time interface state.

[0022] In practical implementation, after acquiring the interface image stream of the target human-machine interface from the train operation monitoring system, the structured display elements in the interface image stream are identified. These structured display elements include digital instrument readings, status indicator icons, text prompts, and operation button areas. The identified structured display elements are compared with entries in a pre-set interface knowledge base. These entries include the expected location of the display element, color coding specifications, text templates, and associated signal names. Through comparison, the functional semantics and data source of each matching structured display element are labeled, and interface description information containing these semantic tags is output. This interface description information is a structured data set recording the semantics and values ​​of "Current speed: 180km / h," "Braking status: Applying," and "Ahead signal: Green light" on the interface.

[0023] In some embodiments, dynamic intent mapping is performed on the interface description information to generate interface state assumptions for the current operating scenario. Semantic markers in the interface description information are parsed to identify the functional component types corresponding to the semantic markers, such as "speed display," "brake status indication," and "signal status." The current train operation mode parameters are obtained, including train control mode, track conditions, and dispatch commands. The operation mode parameters are associated and matched with the functional component types to determine the current operating scenario as "high-speed cruise braking preparation." Based on this current operating scenario, a preset scenario-state mapping rule is queried. This mapping rule defines the set of states the interface should present in this scenario, thereby generating interface state assumptions corresponding to the interface description information. These interface state assumptions include "traction output should be zero," "electric brake icon should be lit," and "safe distance warning line should be displayed."

[0024] Optionally, the system iterates through the assumptions included in the interface state hypothesis, verifying the feasibility of each assumption in the target HMI. This verification process involves checking whether the underlying software components of the target HMI support generating the state described by the assumption, and whether current hardware resources are sufficient. Assumptions with feasibility exceeding a preset threshold are selected; for example, "traction output should be zero" and "electric brake icon should be lit" are confirmed as feasible, while "safe distance warning line should be displayed" is marked as low feasibility due to unavailable sensor data. Based on the selected assumptions with feasibility exceeding the preset threshold, the interface state hypothesis for the current operating scenario is generated.

[0025] It is understandable that the process involves retrieving historical logs to verify the interface state hypothesis and obtain a verified real-time interface state. This involves retrieving historical operation logs associated with the target human-machine interface from the train's central data server. These logs record each operation command and the interface system's response, tagged with timestamps. The process then searches for time-adjacent record segments in the historical operation logs, covering a period before and after the time corresponding to the interface state hypothesis. The interface response data recorded in these segments is extracted, including the actual values ​​displayed on the interface at the historical time, the sequence of state changes, and system messages. This data is then compared one by one with the expected data of the interface state hypothesis. For example, the "actual traction output value" recorded in the historical logs is compared with the hypothesis that "traction output should be zero," recording the consistent and inconsistent items. The total number of consistent items is counted, and when the total reaches a preset verification pass standard, the interface state hypothesis is considered valid. This preset verification pass standard is defined by a proportional formula:

[0026] Where: V represents the verification pass value. This indicates the total number of consistent items. This represents the total number of items compared. When V is greater than or equal to a preset threshold... At that point, the interface state hypothesis is deemed valid. The valid interface state hypothesis is then merged with the latest time point data from the interface response data to form a verified real-time interface state. The verified real-time interface state is a snapshot of the interface state with higher confidence, which integrates the hypothesis logic and historical actual data.

[0027] In one embodiment of the present invention, see [reference] Figure 3 A fixed time window length is set, and based on this time window length, multiple consecutive historical period data are extracted from the historical database corresponding to the real-time interface state. Key attribute values ​​of the real-time interface state are extracted from each historical period data; these key attribute values ​​are arranged in chronological order to form the evolution trajectory of the real-time interface state across multiple historical periods. A baseline behavior pattern curve is set on this evolution trajectory; the difference between the data at each time point on the evolution trajectory and the corresponding point on the baseline behavior pattern curve is calculated, and continuous time regions where the difference exceeds a preset fluctuation range are marked as the pattern deviation interval. The specific process for calculating the stability quantification value of the interface display element based on the pattern deviation interval includes: statistically analyzing the cumulative duration and frequency of occurrence of the pattern deviation interval within the total analysis period; measuring the numerical change amplitude of the interface display element within each pattern deviation interval; obtaining the standard change range of the interface display element under normal working mode; constructing a stability evaluation function based on the cumulative duration, the frequency of occurrence, the numerical change amplitude, and the standard change range; and inputting the cumulative duration, the frequency of occurrence, and the numerical change amplitude into the stability evaluation function to calculate the stability quantification value of the interface display element.

[0028] In practice, a fixed time window length is set, configured as 24 hours. Based on this time window length, multiple consecutive historical period data are extracted from the historical database corresponding to the real-time interface status. The historical period data consists of data blocks containing historical records of the interface status, divided according to the time window length. Within each historical period data, key attribute values ​​of the real-time interface status are extracted. These key attribute values ​​can be the flashing frequency of indicator lights, the sequence of instrument readings, or the average pixel color value of a specific area. The key attribute values ​​extracted from multiple consecutive historical periods are arranged chronologically to form the evolution trajectory of the real-time interface status across multiple historical periods. This evolution trajectory is a discrete sequence with time as the horizontal axis and key attribute values ​​as the vertical axis.

[0029] In some embodiments, a baseline behavior pattern curve is set on the evolution trajectory. This curve is generated by statistically fitting a sequence of similar key attribute values ​​under historical normal operating conditions, representing the expected standard behavior. The difference between the data at each time point on the evolution trajectory and the corresponding point on the baseline behavior pattern curve is calculated. This difference reflects the degree of deviation between the actual observed value and the expected standard value. Continuous time regions where the difference exceeds a preset fluctuation range are marked as pattern deviation intervals. The preset fluctuation range is determined based on the normal operating tolerance of the key attribute values. The pattern deviation intervals indicate the periods during which abnormal or uncertain interface display behavior occurs. Optionally, based on the pattern deviation intervals, a specific process is included to calculate the stability quantification value of the interface display elements. The cumulative duration and frequency of the pattern deviation intervals within the total analysis period are statistically analyzed. The total analysis period is the total time span covered by multiple selected historical periodic data. The numerical change amplitude of the interface display elements within each pattern deviation interval is measured. This amplitude is the difference between the maximum and minimum values ​​of the key attribute values ​​within the pattern deviation interval. The standard variation range of the interface display elements under normal operating conditions is obtained. This standard variation range defines the upper and lower limits of the allowable numerical change of the element under fault-free conditions. A stability evaluation function is constructed based on the cumulative duration, frequency of occurrence, magnitude of numerical change, and standard variation range.

[0030] As can be understood, the cumulative duration, frequency of occurrence, and magnitude of numerical change are input into the stability evaluation function to calculate the quantified stability value of the displayed elements. A specific example of the stability evaluation function is as follows:

[0031] Where: S represents the stability quantization value to be determined, with a value range between 0 and 1, and a higher value indicates greater stability; T represents the cumulative duration of the mode deviation interval; Indicates the total analysis period length; F represents the frequency of model deviation intervals; A represents the magnitude of numerical changes within the model deviation intervals; This indicates the maximum permissible range of variation for the standard. , , These are pre-set weighting coefficients that satisfy... .

[0032] In one embodiment of the present invention, a fault knowledge graph is constructed, with interface display elements as nodes and logical dependencies and signal transmission relationships between elements as edges. In the fault knowledge graph, each node is labeled with a historical stability quantification value range and a corresponding typical fault mode. The calculated stability quantification value of the interface display element is compared with the historical stability quantification value range of the corresponding node in the fault knowledge graph; if the stability quantification value falls within the historical stability quantification value range corresponding to a typical fault mode, the path associated with that typical fault mode in the fault knowledge graph is activated. From the activated paths, paths that meet the current train operation constraints are selected as the potential fault evolution paths. The specific process of reversely deriving the logical conflict nodes in the target human-machine interface based on the potential fault evolution path includes: starting from the terminal fault phenomenon of the potential fault evolution path, tracing back along the edges in the fault knowledge graph; during the reverse tracing process, recording all the nodes passed through and the causal relationships between the nodes; analyzing the causal relationships to identify causal chains with contradictions or unmet conditions; locating the node that the contradictory or unmet causal chains all point to; and determining the node as a logical conflict node in the target human-machine interface.

[0033] In practical implementation, a fault knowledge graph is constructed, with interface display elements as nodes and logical dependencies and signal transmission relationships between elements as edges. Interface display elements include speedometers, brake status indicators, and signal display icons. Logical dependencies are defined as the requirement for one interface display element to output a specific signal from another interface display element for normal display. Signal transmission relationships record the data flow path from the sensor through the processor to the display unit. In the fault knowledge graph, each node is labeled with its historical stability quantification value range and corresponding typical fault modes. The historical stability quantification value range is derived from the statistical summarization of the stability quantification values ​​of the same interface display element in historical normal data and fault cases. Typical fault modes include "display refresh delay," "data packet loss," and "logical deadlock."

[0034] In some embodiments, the calculated stability quantification value of the interface display element is compared with the historical stability quantification value range of the corresponding node in the fault knowledge graph. The comparison operation checks which historical statistical interval the current stability quantification value falls within. If the stability quantification value falls within the historical stability quantification value range corresponding to a typical fault mode, the path associated with the typical fault mode in the fault knowledge graph is activated. The activation operation marks the nodes and edges on the path as pending analysis. From the activated paths, paths that meet the current train operation constraints are selected as potential fault evolution paths. The current train operation constraints include the current train speed range, track gradient information, and onboard equipment model matching rules. Only paths that meet all these constraints are retained.

[0035] Optionally, based on the potential fault evolution path, logical conflict nodes in the target human-machine interface are deduced in reverse. Starting from the terminal fault phenomenon of the potential fault evolution path, the process traces backward along the edges in the fault knowledge graph. The terminal fault phenomenon is the fault state described at the end of the path. During the reverse tracing process, all traversed nodes and the causal relationships between nodes are recorded. These causal relationships describe how the state of an upstream node affects the state of a downstream node. The recorded causal relationships are analyzed to identify causal chains with contradictions or unmet conditions. Contradictions manifest as conflicting input condition requirements for the same node in two causal chains, while unmet conditions manifest as the output state of a node failing to meet the activation conditions of its downstream node. The nodes that contradictory or unmet causal chains commonly point to are located. A common point indicates that the inference endpoints or key influencing points of multiple abnormal causal chains converge on the same interface display element node.

[0036] It is understandable that identifying nodes that point to each other as logically conflicting nodes in the target human-machine interface involves evaluating the node's centrality in the fault knowledge graph and its role in the abnormal causal chain. One method for calculating path confidence matching degree to assist in filtering logically conflicting nodes is as follows:

[0037] Where: C represents the path confidence matching degree of the logically conflicting nodes, M represents the number of abnormal causal chains that point to the same node, and N represents the total number of causal chains involving the node in the fault knowledge graph. It is an indicator function that takes the value 1 when the i-th causal chain is identified as an anomaly, and 0 otherwise. This represents the preset weight of the i-th causal chain, which is set according to the criticality of signal transmission in the chain.

[0038] In one embodiment of the present invention, a simulated operation sequence covering normal operation, boundary operation, and abnormal operation is designed based on the functional definition of the logical conflict node. In a simulation environment, the simulated operation sequence is applied as input to a target human-machine interface containing the logical conflict node. The internal state variables and external output signals of the target human-machine interface are monitored in real time during the processing of the simulated operation sequence. The changes in the internal state variables and the timing waveforms of the external output signals are recorded. Feature data characterizing the dynamic behavior of the target human-machine interface is extracted from the recorded changes and timing waveforms as the response features.

[0039] In practical implementation, simulated operation sequences covering normal operation, boundary operation, and abnormal operation are designed based on the functional definitions of logical conflict nodes. The functional definitions of logical conflict nodes are derived from the system design document, describing the specific data display and update logic that the node is responsible for handling in the train's human-machine interface. The simulated operation sequence is a set of excitation signals applied to the input interfaces associated with the logical conflict nodes, arranged in a predetermined order. The normal operation sequence simulates the driver's input under standard procedures; the boundary operation sequence simulates the input when the input parameters are within the limits of the allowable range; and the abnormal operation sequence simulates the input when the input parameters exceed the allowable range or violate the operation logic. The design process constructs systematic test cases based on the input and output specifications of the logical conflict nodes to stimulate their behavior under different operating conditions.

[0040] In some embodiments, a simulated operation sequence is applied as input to a target human-machine interface (HMI) containing logical conflict nodes in a simulation environment. The simulation environment is an accurate replication of the target HMI software and its dependent train control model on an isolated hardware or virtualization platform. The specific implementation of the train control model involves integrating key parameters from the train operation monitoring system, including train control modes, track conditions, and dispatch commands. These parameters collectively form the basis of the simulated train behavior. The model is implemented as a software module in the simulation environment, accurately replicating the control logic and dynamic response of the actual train. It integrates the train's current operating speed range, track gradient information, and onboard equipment model matching rules and operating constraints to ensure that the model can realistically reflect the train's operating characteristics under specific conditions. Deployed on an isolated hardware or virtualization platform, the train control model safely interacts with the target HMI software, thereby supporting the testing of simulated operation sequences for logical conflict nodes and capturing reliable response characteristics for subsequent analysis. The application process uses the simulation environment's interface injection tool to accurately send each excitation signal in the simulated operation sequence to the corresponding input port of the target HMI according to a preset timing sequence. Real-time monitoring of the target human-machine interface's internal state variables and external output signals during the processing of simulated operation sequences. Internal state variables include intermediate calculation results within logical conflict nodes, buffer queue length, and thread status flags. External output signals refer to the graphics, values, or control commands sent to other systems that are ultimately updated on the display screen.

[0041] Optionally, the changes in internal state variables and the time-series waveforms of external output signals are recorded. The changes are saved as a log format of timestamps and variable values, while the time-series waveforms are saved as sampled data of time and signal amplitude. Feature data characterizing the dynamic behavior of the target human-machine interface is extracted from the recorded changes and time-series waveforms as response features. The extraction process involves filtering the raw data, calculating statistics, and identifying specific patterns. The response feature is a multi-dimensional vector, with each dimension corresponding to a behavioral attribute quantified from the monitoring data.

[0042] It is understandable that, for the systematic design of simulation operation sequences, one practice is to partition the input space based on logically conflicting nodes. Referring to Table 1, a simulation operation sequence design for a logically conflicting node named "Speed ​​Display Controller" is shown.

[0043] Table 1: Simulation Operation Sequence Design Table

[0044] Based on multiple feature values ​​extracted from the response characteristics, a comprehensive response deviation index can be calculated to quantify the degree of behavioral anomalies under simulated operations. The formula for this index is:

[0045] Where: D represents the overall response deviation, and N represents the total number of extracted features. This represents the actual observed value of the i-th feature. This represents the expected value of the i-th feature under normal baseline operation. This represents the standard deviation of the i-th feature under normal baseline operation. The larger the overall response deviation D value, the greater the difference between the response characteristics of the target human-machine interface under the simulated operation sequence and the normal baseline behavior.

[0046] See Figure 4 This is a confidence analysis chart of logical conflict node paths in a train's human-machine interface (HMI). It displays the correlation between the path confidence matching degree, the number of abnormal causal chains, and the total number of causal chains for different logical conflict nodes. It is a core indicator chart in the fault knowledge graph analysis phase. The speed display controller ranks first in both confidence matching degree and the number of abnormal causal chains, and should be the primary target for fault investigation. The path confidence matching degree of a node is positively correlated with the number of abnormal causal chains, reflecting the principle that "the more abnormal causal chains, the more central the node is in the fault path." This chart can assist in the accurate location of faults in the train's HMI, clarify the risk level of different logical conflict nodes, and support resource allocation and priority decisions for fault repair.

[0047] In one embodiment of the present invention, the service anomaly source corresponding to the service feature and the behavior anomaly source corresponding to the behavior feature are located. Based on the service anomaly source and the behavior anomaly source, a multi-dimensional analysis task is constructed, which includes a parallel diagnostic scheme for the target human-machine interface. According to a preset security analysis procedure, the parallel diagnostic scheme is sequentially arranged to generate an executable analysis instruction list. The analysis instruction list is sequentially loaded into the simulation test environment where the target human-machine interface is located, driving the target human-machine interface to perform a full-process simulation. After the full-process simulation is completed, the final output result of the target human-machine interface is collected. The specific process of determining the conformity of the final output result and forming a complete intelligent analysis conclusion for the target human-machine interface in conjunction with the multi-dimensional analysis task includes: parsing the interface display content, response time, and data accuracy indicators from the final output result; comparing the interface display content, response time, and data accuracy indicators with the requirements in the predefined system specifications one by one; based on the comparison results, giving a judgment conclusion of conformity, non-conformity, or deviation for each indicator; summarizing the judgment conclusions of all indicator items and combining them with the service anomaly source and behavior anomaly source information discovered during the execution of the multi-dimensional analysis task; and generating a structured diagnostic report by combining the judgment conclusions and the service anomaly source and behavior anomaly source information, which is the complete intelligent analysis conclusion of the target human-machine interface.

[0048] In practical implementation, the service anomaly source corresponding to the service characteristics and the behavior anomaly source corresponding to the behavior characteristics are identified. The service anomaly source refers to the underlying software module, communication link, or hardware component that causes the service characteristics to deviate from the normal mode. The behavior anomaly source refers to the sequence of operation instructions, user input patterns, or external interaction events that cause the behavior characteristics to exhibit abnormal patterns. Based on the service anomaly source and the behavior anomaly source, a multi-dimensional analysis task is constructed. The multi-dimensional analysis task includes a parallel diagnostic scheme for the target human-machine interface. The parallel diagnostic scheme is designed to simultaneously perform root cause investigation and impact assessment on the identified anomaly sources from both the software service layer and the user interaction layer. In some embodiments, the parallel diagnostic scheme is sequentially arranged according to a preset security analysis procedure to generate an executable list of analysis instructions. The security analysis procedure defines the execution priority, resource isolation requirements, and security boundaries for fault injection that the diagnostic operations must follow. The sequential arrangement process, according to the priority rules in the security analysis procedure, prioritizes diagnostic steps involving core service security and arranges diagnostic steps that cause system disturbances in resource-isolated time periods, generating a sequentially arranged list of analysis instructions containing specific control commands and parameters. The analysis instruction list is loaded sequentially into the simulation test environment where the target HMI is located, driving the target HMI to perform a full-process simulation. After the full-process simulation is completed, the final output results of the target HMI are collected. The final output results include a static screenshot of the interface at the end of the simulation process, system event logs for the entire simulation cycle, and performance monitoring data.

[0049] Optionally, a conformity assessment is performed on the final output results. Combined with multi-dimensional analysis tasks, a complete intelligent analysis conclusion for the target human-machine interface is formed. The interface display content, response time, and data accuracy indicators are extracted from the final output results. The interface display content refers to the final state and values ​​of each graphic element on the screen; the response time is the time delay from triggering the diagnostic operation to the expected change in the interface; and the data accuracy indicator is the statistical error between the interface display value and the baseline true value injected into the simulation test environment. The extracted interface display content, response time, and data accuracy indicators are compared one by one with the requirements in the predefined system specifications. The system specifications, in document form, stipulate the functional and performance standards that the target human-machine interface must meet in all test scenarios.

[0050] Understandably, based on the comparison results, a judgment conclusion is given for each indicator item as compliant, non-compliant, or having deviations. This judgment conclusion is derived from a direct comparison logic between the measured value of the indicator and the threshold in the specification. The judgment conclusions for all indicator items are summarized, and combined with service anomaly source and behavior anomaly source information discovered during the multi-dimensional analysis task. Service anomaly source information includes the identifier and error code of the abnormal module, while behavior anomaly source information includes the sequence and context of abnormal operations. By combining the judgment conclusions and service and behavior anomaly source information, a structured diagnostic report is generated. This structured diagnostic report represents the complete intelligent analysis conclusion for the target human-machine interface. The generation process of the complete intelligent analysis conclusion introduces a compliance quantification evaluation function, the expression of which is:

[0051] Where: Q represents the comprehensive compliance quantitative assessment value, and K represents the total number of indicators compared. This represents the measured value of the j-th indicator item. This represents the standard value or allowable range of the j-th indicator item in the system specification, and the function... It is a comparison function, when Fully compatible Output 1 if the requirement is met, 0 if the requirement is not met, and a value between 0 and 1 if there is a deviation. It is the preset weight coefficient of the j-th indicator item and satisfies .

[0052] See Figure 5 This is a comparison chart of the stability scores of train human-machine interface elements. It is used to show the differences between the measured stability values ​​and standard values ​​of different interface elements and is the core analytical chart for interface stability assessment. Data processing and safety monitoring show good stability, and the display panel is close to meeting the standards. The stability of the communication module and user interaction does not meet the standards and is a key area for optimization. The measured values ​​of the communication module and user interaction differ significantly from the standard values, and their operational anomalies should be investigated first. This chart can be used for stability assessment and optimization decisions of train human-machine interfaces, clearly identifying the performance shortcomings of each functional module and supporting continuous improvement of interface reliability.

[0053] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0054] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent analysis method for train human-machine interface display based on artificial intelligence, characterized in that, The method includes: The system acquires the interface image stream of the target human-machine interface from the train operation monitoring system, outputs interface description information containing semantic tags, and obtains the verified real-time interface status. Extract the evolution trajectory of the real-time interface state in multiple historical periods, identify the pattern deviation interval in the evolution trajectory, and calculate the stability quantification value of the interface display elements based on the pattern deviation interval. The stability quantification value is input into the fault knowledge graph to match potential fault evolution paths. Based on the potential fault evolution paths, the logical conflict nodes in the target human-machine interface are deduced in reverse. A simulated operation sequence is applied to the logical conflict node to capture the response features of the target human-machine interface, and service features generated by the underlying display service and behavioral features generated by user operation behavior are separated from the response features. The service features and the behavior features are traced independently to locate the service anomaly source and the behavior anomaly source. Multi-dimensional analysis tasks are constructed, and parallel diagnostic schemes are arranged sequentially according to the security analysis procedure. An analysis instruction list is generated, loaded into the simulation test environment to drive the full process simulation, and the final output results are collected. The final output result is evaluated for compliance, and combined with the multi-dimensional analysis task, a complete intelligent analysis conclusion of the target human-computer interface is formed. The step of extracting the evolution trajectory of the real-time interface state over multiple historical periods and identifying the pattern deviation intervals in the evolution trajectory includes: Set a fixed time window length, and extract multiple consecutive historical period data from the historical database corresponding to the real-time interface status based on the time window length; Extract the key attribute values ​​of the real-time interface state from each of the historical period data; The key attribute values ​​are arranged in chronological order to form the evolution trajectory of the real-time interface state in multiple historical periods. A baseline behavior pattern curve is established on the evolution trajectory; Calculate the difference between the data at each time point on the evolution trajectory and the corresponding point on the baseline behavior pattern curve, and mark the continuous time area where the difference exceeds the preset fluctuation range as the pattern deviation interval.

2. The intelligent analysis method for train human-machine interface display based on artificial intelligence as described in claim 1, characterized in that, The process of acquiring the interface image stream of the target human-machine interface from the train operation monitoring system, outputting interface description information containing semantic tags, and obtaining the verified real-time interface state includes: Identify the structured display elements in the interface image stream, compare the structured display elements with entries in a preset interface knowledge base, and output interface description information containing semantic tags; Dynamic intent mapping is performed on the interface description information to generate an interface state hypothesis under the current operation scenario, and the interface state hypothesis is verified by calling historical logs to obtain the verified real-time interface state. The step of dynamically mapping the interface description information to generate interface state assumptions for the current operation scenario includes: Parse the semantic tags in the interface description information and identify the functional component types corresponding to the semantic tags; Obtain the current train's operating mode parameters, associate and match the operating mode parameters with the functional component types, and determine the current operating scenario; Based on the current operation scenario, a preset scenario-state mapping rule is queried to generate an interface state hypothesis corresponding to the interface description information; Iterate through the assumptions contained in the interface state assumptions and check the feasibility of each assumption in the target human-computer interface. Hypothetical conditions whose feasibility exceeds a preset threshold are selected, and based on the hypothetical conditions, interface state hypotheses for the current operation scenario are generated.

3. The intelligent analysis method for train human-machine interface display based on artificial intelligence as described in claim 2, characterized in that, The process of calling historical logs to verify the interface state assumption and obtain a verified real-time interface state includes: Retrieve the historical operation logs associated with the target human-machine interface from the train's central data server; Search the historical operation log for record segments that are temporally adjacent to the assumed interface state. Extract the interface response data recorded in the recorded segment; The interface response data is compared one by one with the expected data of the interface state assumption, and the consistent items and inconsistent items are recorded. The total number of the consistent items is counted. When the total number reaches the preset verification pass standard, the interface state assumption is determined to be valid. The interface state assumption is fused with the latest time data in the interface response data to form the verified real-time interface state.

4. The intelligent analysis method for train human-machine interface display based on artificial intelligence as described in claim 1, characterized in that, The step of calculating the stability quantification value of the elements displayed on the interface based on the pattern deviation interval includes: The cumulative duration and frequency of the deviation intervals of the described pattern within the total analysis period are statistically analyzed. Measure the magnitude of the numerical change of the displayed element within each deviation interval of the stated pattern; Obtain the standard variation range of the interface display elements in normal working mode; Based on the cumulative duration, the frequency of occurrence, the magnitude of numerical change, and the standard range of change, a stability evaluation function is constructed. The cumulative duration, the frequency of occurrence, and the magnitude of numerical change are input into the stability evaluation function to calculate the stability quantification value of the interface display element.

5. The intelligent analysis method for train human-machine interface display based on artificial intelligence as described in claim 1, characterized in that, The step of inputting the stability quantification value into the fault knowledge graph to match potential fault evolution paths includes: Construct a fault knowledge graph with interface display elements as nodes and logical dependencies and signal transmission relationships between elements as edges; In the fault knowledge graph, each node is labeled with its historical stability quantification value range and corresponding typical fault mode; The calculated stability quantification value of the interface display element is compared with the historical stability quantification value range of the corresponding node in the fault knowledge graph. If the stability quantification value falls within the range of historical stability quantification values ​​corresponding to a typical failure mode, then the path associated with the typical failure mode in the failure knowledge graph is activated. From the activated paths, paths that meet the current train operation constraints are selected as the potential fault evolution paths.

6. The intelligent analysis method for train human-machine interface display based on artificial intelligence as described in claim 1, characterized in that, The step of reversely deriving the logical conflict nodes in the target human-machine interface based on the potential fault evolution path includes: Starting from the terminal fault phenomenon of the potential fault evolution path, trace back in reverse along the edges in the fault knowledge graph; During the reverse tracing process, all nodes visited and the causal relationships between nodes are recorded; Analyze the causal relationships to identify causal chains that contain contradictions or whose conditions are not met; Locate the node that the contradictory or unmet causal chains all point to; The node is identified as a logical conflict node in the target human-machine interface.

7. The intelligent analysis method for train human-machine interface display based on artificial intelligence as described in claim 1, characterized in that, The step of applying a simulated operation sequence to the logical conflict node and capturing the response characteristics of the target human-machine interface includes: Based on the functional definition of the logical conflict nodes, design a simulated operation sequence that covers normal operation, boundary operation, and abnormal operation; In a simulation environment, the simulated operation sequence is used as input and applied to the target human-machine interface containing the logical conflict node; Real-time monitoring of the target human-machine interface's internal state variables and external output signals during the processing of the simulated operation sequence; Record the change process of the internal state variables and the timing waveform of the external output signal; Feature data that characterizes the dynamic behavior of the target human-machine interface is extracted from the recorded change process and the time-series waveform, and used as the response features.

8. The intelligent analysis method for train human-machine interface display based on artificial intelligence as described in claim 1, characterized in that, The process involves independently tracing the service characteristics and behavioral characteristics to pinpoint the sources of service and behavioral anomalies, constructing multi-dimensional analysis tasks, sequentially arranging parallel diagnostic schemes according to security analysis procedures, generating an analysis instruction list, loading it into a simulation test environment to drive the entire process simulation, and collecting the final output results, including: Locate the service anomaly source corresponding to the service feature, and the behavior anomaly source corresponding to the behavior feature; Based on the service anomaly source and the behavior anomaly source, a multi-dimensional analysis task is constructed, which includes a parallel diagnostic scheme for the target human-machine interface. Based on the preset security analysis procedure, the parallel diagnostic scheme is sequentially arranged to generate an executable analysis instruction list; The analysis instruction list is sequentially loaded into the simulation test environment where the target human-machine interface is located, driving the target human-machine interface to perform a full-process simulation. After the full-process simulation is completed, the final output result of the target human-machine interface is collected.

9. The intelligent analysis method for train human-machine interface display based on artificial intelligence as described in claim 8, characterized in that, The process of determining the conformity of the final output result, combined with the multi-dimensional analysis task, forms a complete intelligent analysis conclusion for the target human-computer interface, including: The final output results are used to parse the interface display content, response time, and data accuracy indicators. The interface display content, response time, and data accuracy indicators are compared one by one with the requirements in the predefined system specifications. Based on the comparison results, a judgment conclusion is given for each indicator item: it meets the requirements, it does not meet the requirements, or there is a deviation. Summarize the judgment conclusions of all indicator items, and combine them with the information of the service anomaly source and the behavior anomaly source found during the execution of the multi-dimensional analysis task; Based on the combined judgment conclusions and the information on the service anomaly sources and behavior anomalies, a structured diagnostic report is generated. This structured diagnostic report represents the complete intelligent analysis conclusions of the target human-machine interface.

Images