Operation and maintenance fault identification and treatment scheme generation method, system, device and medium

Abstract

The application discloses a kind of operation and maintenance fault identification and treatment scheme generation method, system, equipment and medium, belong to rail transit intelligent operation and artificial intelligence technical field.The method includes: obtaining fault image data, constructs motor train unit global operation and maintenance fault identification benchmark data set;Multi-scale visual diagnostic architecture of local defect perception, global state perception and time sequence change modeling branch collaborative fusion is constructed and is trained;Special field corpus of rail transit is constructed;Large model in operation and maintenance field is trained based on hierarchical fault knowledge distillation instruction fine-tuning mechanism;The recognition result is obtained by inputting field fault image into multi-scale visual diagnostic architecture and is mapped to unified fault semantic space;Fault semantic information and historical operation and maintenance log and user prompt word are jointly input into large model in operation and maintenance field, and fault treatment scheme, maintenance process and spare parts replacement suggestion are automatically generated.The application realizes motor train unit fault accurate identification, cause analysis and treatment scheme automatic generation.

Description

Technical Field

[0001] This invention belongs to the field of intelligent operation and maintenance and artificial intelligence technology for rail transit, and in particular relates to a method, system, equipment and medium for generating operation and maintenance fault identification and handling solutions. Background Technology

[0002] With the continuous expansion of the high-speed railway network and the increasing complexity of EMU (Electric Multiple Unit) equipment, the operation and maintenance model of rail transit is gradually evolving from traditional planned maintenance and experience-based maintenance to intelligent sensing, precise diagnosis, and autonomous decision-making. As a complex piece of equipment integrating mechanical, electrical, control, and communication systems, the EMU involves numerous critical components during operation, including pantographs, bogies, braking systems, door systems, car body structures, and onboard electrical equipment. If these components experience wear, cracks, loosening, deformation, corrosion, or functional failure, it will not only reduce the reliability and efficiency of vehicle operation but, in severe cases, may also lead to train delays, equipment damage, or even operational safety accidents. Therefore, how to achieve rapid identification, accurate diagnosis, and efficient handling of EMU faults has become an important research direction in the field of intelligent operation and maintenance of rail transit.

[0003] In recent years, with the development of computer vision technology, some progress has been made in using image recognition methods for the condition detection of high-speed train equipment. By deploying visual acquisition equipment in maintenance depots, depots, and onboard monitoring systems, a large number of images of key components in operation can be obtained, providing a data foundation for automatic fault detection. However, the operation and maintenance scenario of high-speed trains is characterized by a wide variety of components, complex fault modes, and significant changes in the operating environment. This leads to problems such as changes in lighting, occlusion interference, complex backgrounds, and large differences in the scale of fault targets in the actual acquired images. Most existing visual detection methods use a single convolutional network or a single Transformer structure for feature extraction, which makes it difficult to simultaneously take into account local defect details, global state changes, and long-term fault evolution patterns, thus limiting the accuracy and robustness of fault identification in complex scenarios.

[0004] On the other hand, existing fault diagnosis systems typically separate visual inspection from maintenance decision-making processes. They only output detection results such as fault type or location, still requiring manual review of maintenance procedures, historical cases, and expert experience to complete subsequent fault analysis and maintenance decisions. This approach is not only inefficient, but also prone to inconsistencies in maintenance plans due to differences in experience among different maintenance personnel, failing to meet the demands of modern intelligent rail transit operation and maintenance for standardized, automated, and intelligent decision-making. Furthermore, while rapidly developing general-purpose large language models possess strong natural language understanding and generation capabilities, their lack of rail transit expertise, fault mechanism knowledge, and maintenance decision-making experience makes them prone to problems such as insufficient professional knowledge, incomplete reasoning logic, and insufficient engineering feasibility of generated results when directly applied to EMU (Electric Multiple Unit) operation and maintenance scenarios. Summary of the Invention

[0005] This invention aims to solve the above-mentioned technical problems and provides a method, system, equipment, and medium for generating operation and maintenance fault identification and handling solutions. By integrating fault images of key components of EMU trains, operation and maintenance knowledge, and historical maintenance experience information, it establishes an intelligent operation and maintenance technology system that integrates intelligent fault perception, fault semantic understanding, fault mechanism reasoning, and maintenance decision generation. This system enables accurate fault identification, cause analysis, and automatic generation of handling solutions in complex operation and maintenance scenarios of EMU trains, providing technical support for intelligent operation and maintenance of rail transit equipment, optimization of maintenance decisions, fault risk early warning, and intelligent application of operation and maintenance knowledge.

[0006] To achieve the above-mentioned objectives, this invention provides a method for generating operation and maintenance fault identification and handling solutions, comprising the following steps: Step 1: Use fixed visual acquisition devices deployed in key operation and maintenance areas of EMUs to acquire fault image data and construct a benchmark dataset for EMU full-domain operation and maintenance fault identification. Step 2: Construct a multi-scale visual diagnostic architecture based on the collaborative fusion of local defect perception, global state perception, and temporal change modeling branches, and train, optimize, and update the parameters of the benchmark dataset; Step 3: Collect EMU operation and maintenance procedures, maintenance records, fault case database, expert diagnosis reports, and historical operation and maintenance logs to build a corpus for the rail transit specific domain; Step 4: Train the large model in the operation and maintenance field based on the hierarchical fault knowledge distillation instruction fine-tuning mechanism to enable it to have the ability to understand faults, reason about causes, and generate maintenance decisions. Step 5: Input the fault images collected on site into the multi-scale visual diagnostic architecture to obtain image recognition results containing fault category, fault location and fault evolution status information, and map them to a unified fault semantic space; Step 6: Integrate the fault semantic information obtained in Step 5 with the historical operation and maintenance logs, historical fault records, and operation and maintenance requirement prompts entered by the user for the corresponding device, and input them together into the operation and maintenance domain big model trained in Step 4 to automatically generate corresponding fault handling solutions, maintenance processes, and spare parts replacement suggestions.

[0007] Furthermore, step 1 is implemented in the following way: Step 1.1: Use a fixed industrial camera to continuously acquire images of the EMU bogie, pantograph, door system, braking system, and car body structure. Step 1.2: Extract keyframes from the video stream every 20 frames at a preset time interval; Step 1.3: Construct a fault label set by using intelligent assisted labeling as the main method and manual labeling, verification, and calibration as a supplementary method; Step 1.4: Establish a three-dimensional annotation system that includes fault category labels, fault area labels, and fault level labels; Step 1.5: Construct the training set, validation set, and test set in a ratio of 8:1:2.

[0008] Furthermore, the calculation formulas for the local defect perception branch, global state perception branch, and temporal change modeling branch in step 2 are as follows: In the formula, For input image, This represents operation based on channel connection. The window size is Deformable convolution, It is a deformable convolution with a window size of 5×5. It is a deformable convolution with a window size of 7×7; q and k Represents query and key, Input image generated through a linear layer A copy, Represents matrix multiplication. for q 1. k 1 and q 2. k 2. Calculate the difference after matrix multiplication; It is a gated structure consisting of several convolutional layers, normalization layers, and activation layers, with the last layer using the GeLU activation function. Represents element-wise multiplication; This represents a selective scanning mechanism path controlled by variance; , and These refer to the results of the local defect perception branch, the global state perception branch, and the temporal change modeling branch, respectively.

[0009] Furthermore, the branch collaborative fusion in step 2 is , and Information fusion is achieved through a cross-branch dynamic coupling module, and its calculation formula is as follows: In the formula, , and These represent trainable parameters, initialized to 0.5, 0.25, and 0.25 respectively. This represents the result of fusing information from different branches.

[0010] Furthermore, step 3 is implemented in the following way: Step 3.1: Collect the EMU operation and maintenance procedures, maintenance operation instructions, equipment technical manuals, fault case library, historical maintenance work orders, expert diagnosis reports and historical operation and maintenance logs to form the original operation and maintenance text data pool. Step 3.2: Perform format unification processing on the original operation and maintenance text data pool to eliminate format differences between data from different sources, and complete text cleaning, duplicate content removal, garbled character correction, and invalid field filtering. Step 3.3: Establish domain entity recognition rules based on the rail transit professional terminology dictionary, and automatically extract fault phenomena, faulty components, fault causes, maintenance measures, spare parts names, detection parameters and operating environment information from the text to form a standardized set of operation and maintenance entities; Step 3.4: Establish entity associations based on the operation and maintenance business logic, map the fault phenomena, fault causes, faulty components and maintenance measures, construct a fault knowledge link, and realize the complete association from fault manifestation to maintenance and handling process; Step 3.5: Perform time series correlation analysis on the operation and maintenance records generated at different operating stages for the same fault, extract information on the entire process of fault occurrence, fault expansion, fault deterioration and fault handling, and form a fault evolution knowledge sample; Step 3.6: Construct a knowledge graph for rail transit operation and maintenance using fault knowledge links and fault evolution knowledge samples, and organize and manage key components of EMU trains, typical fault modes, fault propagation paths and maintenance strategies in a unified manner; Step 3.7: Automatically generate fault description question-and-answer pairs, fault cause analysis question-and-answer pairs, maintenance process question-and-answer pairs, and maintenance decision question-and-answer pairs based on the knowledge graph to form a multi-level operation and maintenance instruction dataset; Step 3.8: Combining the structured operation and maintenance knowledge, fault evolution knowledge, and operation and maintenance instruction dataset obtained in steps 3.3 to 3.7, construct a corpus for the rail transit domain, providing basic data support for subsequent knowledge injection, instruction fine-tuning, and inference ability training of large-scale operation and maintenance models.

[0011] Furthermore, step 4, which trains the large model of the operation and maintenance domain based on the hierarchical fault knowledge distillation instruction fine-tuning mechanism, constructs a unified fault reasoning chain from fault phenomena, fault mechanisms, fault propagation paths, and maintenance and handling solutions, and realizes the transfer of domain knowledge to the parameter space of the large model through a multi-stage knowledge alignment method.

[0012] Furthermore, the construction of the fault reasoning chain and the transfer of domain knowledge to the large model parameter space are achieved through the following methods: First, fault reasoning samples are generated using the rail transit-specific domain corpus constructed in step 3. The calculation formula is as follows: In the formula, This describes the fault symptoms. Indicates the fault formation mechanism, Indicates the fault propagation path, Indicate the repair and handling plan, Indicates the number of training samples; Subsequently, by constructing a fault link mapping relationship and a fault link attention alignment mechanism, the model learns the causal relationship between fault states and strengthens the model's understanding of the train fault evolution process. At the same time, a unified training objective function is constructed by introducing loss functions of multiple dimensions, and its calculation formula is as follows: In the formula, Indicates the first A fault reasoning chain, This represents the fault chain coding function. and These represent the fault query vector and the key-value vector, respectively. Representing feature dimension, This is a fault association matrix that assigns weights λ to entity association edges based on association frequency, causal strength, and expert experience. Used to weight and enhance the features of operation and maintenance text. For the total loss function, To monitor and fine-tune losses, For fault knowledge distillation loss, To diagnose path consistency loss, Loss due to the feasibility of the repair plan. For failure chain constraint loss, , , and This represents the corresponding weight parameter.

[0013] Furthermore, the aforementioned and Use the balanced cross-entropy loss function Use consistency loss or absolute value loss. L2 loss and cosine similarity loss are used for collaborative supervision. , , and All are initialized to floating-point numbers between 0 and 1.

[0014] Furthermore, step 5, mapping to a unified fault semantic space, is achieved in the following way: the visual diagnostic results are associated and matched with standard fault terms, key component information, and maintenance knowledge in the rail transit operation and maintenance knowledge base using the fault semantic mapping module, and converted into unified fault semantic information, so that the image recognition results can be effectively connected with the subsequent text reasoning process.

[0015] Furthermore, step 6, which automatically generates corresponding fault handling solutions, maintenance procedures, and spare parts replacement suggestions, is achieved by: combining the fault category, fault location, and fault evolution status in the fault semantic information of the large-scale model in the operation and maintenance domain, calling on maintenance experience and handling records in historical operation and maintenance cases, reasoning and analyzing the causes of fault formation, potential impact range, and possible propagation paths, and automatically generating targeted fault handling solutions, maintenance operation procedures, inspection item suggestions, and spare parts replacement suggestions in combination with the current operation and maintenance scenario.

[0016] This invention provides a system for generating operation and maintenance fault identification and handling solutions, used to implement the above method, including: The image acquisition module is used to acquire fault image data using fixed vision acquisition devices deployed in key maintenance areas of EMU trains; The benchmark dataset construction module is used to construct a benchmark dataset for the identification of faults in the entire operation and maintenance of high-speed trains based on the fault image data acquired by the image acquisition module. The multi-scale visual diagnosis module includes a local defect perception branch, a global state perception branch, a temporal change modeling branch, and a cross-branch dynamic coupling module. It is used to perform multi-branch collaborative diagnosis on the input fault image and output image recognition results containing fault category, fault location, and fault evolution state information. The corpus construction module is used to collect EMU operation and maintenance procedures, maintenance records, fault case database, expert diagnosis reports and historical operation and maintenance logs to build a corpus for the rail transit domain. The large-scale model in the operation and maintenance field is trained based on a hierarchical fault knowledge distillation instruction fine-tuning mechanism, and has the ability to understand faults, reason about causes, and generate maintenance decisions. The fault semantic mapping module is used to map the image recognition results output by the multi-scale visual diagnosis module to a unified fault semantic space to form fault semantic information. The solution generation module is used to input fault semantic information, historical operation and maintenance log information of the corresponding equipment, and user prompts into the operation and maintenance domain big model, and automatically generate corresponding fault handling solutions, maintenance processes, and spare parts replacement suggestions.

[0017] Furthermore, the fault semantic mapping module includes: The fault entity parsing unit is used to extract fault category, fault location and fault status information from the visual diagnostic results; The knowledge association matching unit is used to establish semantic associations between visual diagnostic results and the rail transit operation and maintenance knowledge base; The fault profile generation unit is used to generate fault semantic description results in a unified format.

[0018] An electronic device includes: one or more processors; and a storage device for storing one or more programs; when the programs are executed by the one or more processors, the processors cause the processors to perform the methods described above.

[0019] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0020] The beneficial effects of this application are as follows: This application proposes a generative AI-driven method for generating fault identification and handling solutions for the entire operation and maintenance of high-speed trains. By constructing a multi-scale visual diagnostic architecture, it achieves accurate perception and intelligent identification of fault characteristics in key components of high-speed trains. By constructing a corpus specifically for rail transit and a hierarchical fault knowledge distillation instruction fine-tuning mechanism, it achieves efficient transfer of operation and maintenance knowledge to a large model and enhances professional reasoning capabilities. Through a fault semantic mapping module, it establishes semantic associations between visual diagnostic results and operation and maintenance knowledge, realizing the conversion of fault image information into operation and maintenance knowledge expressions. By integrating fault semantic information, historical operation and maintenance log information, and user prompts into a large model in the operation and maintenance domain, it automatically generates targeted fault handling solutions, maintenance procedures, and spare parts replacement suggestions, achieving integrated intelligent processing of fault identification, fault analysis, and maintenance decision-making. This method can effectively reduce reliance on manual diagnosis, improve the efficiency of high-speed train operation and maintenance, the accuracy of fault handling, and the consistency of maintenance decisions, providing technical support for the intelligent, digital, and autonomous operation and maintenance of rail transit equipment. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the exemplary embodiments of this application, the advantages of the foregoing and / or additional aspects of this application will become clearer and easier to understand after the embodiments are described in conjunction with the following drawings. Obviously, the drawings presented herein are only the drawings corresponding to some embodiments to be described in this invention, and not all drawings; for those skilled in the art, other drawings can still be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a generative AI-driven method for generating fault identification and handling solutions for the entire operation and maintenance of high-speed trains, according to an embodiment of this application. Figure 2 This is a flowchart illustrating a multi-scale visual diagnostic architecture. Figure 3 This is a flowchart illustrating the hierarchical fault knowledge distillation instruction fine-tuning mechanism. Detailed Implementation

[0023] To more clearly grasp the foregoing objectives, features and advantages of this application, the present invention will be explained in detail below with reference to the accompanying drawings and corresponding embodiments. However, this does not limit the scope of protection of the present invention to the scope described in the embodiments.

[0024] Figure 1 This is a flowchart illustrating a generative AI-driven method for generating fault identification and handling solutions for the entire operation and maintenance of high-speed trains, provided as an embodiment of this application. This example takes multi-dimensional information as input and obtains targeted fault handling and diagnosis solutions by constructing a multi-scale visual diagnostic architecture and a hierarchical fault knowledge distillation instruction fine-tuning mechanism.

[0025] like Figure 1 As shown, the method in this embodiment specifically includes the following steps: Step 1: Use fixed visual acquisition devices deployed in key operation and maintenance areas of EMUs to acquire fault image data and construct a benchmark dataset for EMU full-domain operation and maintenance fault identification.

[0026] The fixed visual acquisition device can be deployed in various maintenance scenarios, such as maintenance depots, EMU depots, undercarriage inspection lines, pantograph inspection areas, door inspection areas, and bogie inspection areas. The visual acquisition device can employ industrial cameras, line scan cameras, network cameras, or other vision equipment with image acquisition capabilities to obtain image information of key EMU components under different operating conditions.

[0027] Specifically, continuous image acquisition is performed on key components of the EMU using a fixed visual acquisition device to obtain raw image data covering normal conditions and various fault states. These key components include, but are not limited to, pantographs, bogies, braking systems, door systems, car body structures, onboard electrical equipment, and undercarriage running gear. Furthermore, to ensure the quality of subsequent model training, the raw images undergo data filtering and preprocessing, including image denoising, size standardization, brightness correction, contrast enhancement, and outlier filtering, to reduce the impact of complex environmental factors on model training. Subsequently, experts in the rail transit field manually annotate the image data to form a standard fault labeling system. The label information includes fault category, fault location, fault level, and fault evolution stage. Finally, the training set, validation set, and test set are divided according to a preset ratio, and a benchmark dataset for EMU full-domain operation and maintenance fault identification is constructed, providing a data foundation for subsequent multi-scale visual diagnostic architecture training. This step includes: Step 1.1: Use a fixed industrial camera to continuously acquire images of the EMU bogie, pantograph, door system, braking system, and car body structure. Step 1.2: Extract keyframes from the video stream every 20 frames at a preset time interval; Step 1.3: Construct a fault label set by using intelligent assisted labeling as the main method and manual labeling, verification, and calibration as a supplementary method; Step 1.4: Establish a three-dimensional annotation system that includes fault category labels, fault area labels, and fault level labels; Step 1.5: Based on the data obtained in steps 1.1, 1.2, 1.3 and 1.4, construct the training set, validation set and test set in a ratio of 8:1:2.

[0028] Step 2: Construct a multi-scale visual diagnostic architecture based on the collaborative fusion of local defect perception, global state perception, and temporal change modeling branches, and train, optimize, and update the parameters of the benchmark dataset.

[0029] Specifically, such as Figure 2 As shown, this embodiment constructs a multi-scale visual diagnostic architecture for the entire operation and maintenance scenario of high-speed trains. This architecture consists of a local defect perception branch, a global state perception branch, a temporal change modeling branch, and a cross-branch feature fusion module. Through collaborative modeling of information at different scales, levels, and semantic spaces, it achieves accurate extraction and intelligent diagnosis of fault features of key components of high-speed trains. The calculation formula is as follows: In the formula, For input image, This represents operation based on channel connection. The window size is Deformable convolution, It is a deformable convolution with a window size of 5×5. It is a deformable convolution with a window size of 7×7; q and k Represents query and key, Input image generated through a linear layer A copy, Represents matrix multiplication. for q 1. k 1 and q 2. k 2. Calculate the difference after matrix multiplication; It is a gated structure consisting of several convolutional layers, normalization layers, and activation layers, with the last layer using the GeLU activation function. Represents element-wise multiplication; This represents a selective scanning mechanism path controlled by variance. By calculating the strength of the variance, it scans the content in the image layer by layer in descending order of variance. It should be noted that... The generation process allows the model to perceive the feature differences between the two independent similarity matrix generation processes, and improves the dimensionality information of this differentiated representation through a gating mechanism, ultimately benefiting the capture of the global state of the input image. Among these, , and These refer to the results of the local defect perception branch, the global state perception branch, and the temporal change modeling branch, respectively.

[0030] , and Information fusion is achieved through a cross-branch dynamic coupling module, and its calculation formula is as follows: In the formula, , and These represent trainable parameters, initialized to 0.5, 0.25, and 0.25 respectively. This represents the result of fusing information from different branches. This multi-scale visual diagnostic architecture consists of several layers of multi-branched and coupled structures with different scales, and the output of the last layer is the final result of the visual diagnostic architecture. Under the combined effect of the balanced cross-entropy loss function and cross-union ratio loss function, the truth graph will be continuously supervised and guided.

[0031] It should be noted that the multi-scale visual diagnostic architecture is trained using the EMU full-domain operation and maintenance fault identification benchmark dataset constructed in step 1. The network parameters are continuously updated through the backpropagation algorithm, gradually converging the local defect perception capability, global state analysis capability, and fault evolution modeling capability to the optimal state. After training, the multi-scale visual diagnostic architecture can output diagnostic results such as fault category, fault location, fault level, and fault evolution stage, providing fundamental support for subsequent fault semantic description and maintenance decision generation.

[0032] Step 3: Collect EMU operation and maintenance procedures, maintenance records, fault case database, expert diagnostic reports, and historical operation and maintenance logs to build a corpus for the rail transit field.

[0033] Specifically, to enable the subsequent large-scale operation and maintenance model to possess the understanding and reasoning capabilities of rail transit professional knowledge, this embodiment constructs a dedicated rail transit domain corpus for the entire operation and maintenance scenario of high-speed trains. This step includes: Step 3.1: Collect the EMU operation and maintenance procedures, maintenance operation instructions, equipment technical manuals, fault case library, historical maintenance work orders, expert diagnosis reports and historical operation and maintenance logs to form the original operation and maintenance text data pool. Step 3.2: Perform format unification processing on the original operation and maintenance text data pool to eliminate format differences between data from different sources, and complete text cleaning, duplicate content removal, garbled character correction and invalid field filtering; Step 3.3: Establish domain entity recognition rules based on the rail transit professional terminology dictionary, and automatically extract fault phenomena, faulty components, fault causes, maintenance measures, spare parts names, detection parameters and operating environment information from the text to form a standardized set of operation and maintenance entities; Step 3.4: Establish entity associations based on the operation and maintenance business logic, map the fault phenomena, fault causes, faulty components and maintenance measures, construct a fault knowledge link, and realize the complete association from fault manifestation to maintenance and handling process; Step 3.5: Perform time series correlation analysis on the operation and maintenance records generated at different operating stages for the same fault, extract information on the entire process of fault occurrence, fault expansion, fault deterioration and fault handling, and form a fault evolution knowledge sample; Step 3.6: Construct a knowledge graph for rail transit operation and maintenance using fault knowledge links and fault evolution knowledge samples, and organize and manage key components of EMU trains, typical fault modes, fault propagation paths and maintenance strategies in a unified manner; Step 3.7: Automatically generate fault description question-and-answer pairs, fault cause analysis question-and-answer pairs, maintenance process question-and-answer pairs, and maintenance decision question-and-answer pairs based on the knowledge graph to form a multi-level operation and maintenance instruction dataset; Step 3.8: Combining the structured operation and maintenance knowledge, fault evolution knowledge, and operation and maintenance instruction dataset obtained in steps 3.3 to 3.7, construct a corpus for the rail transit domain, providing basic data support for subsequent knowledge injection, instruction fine-tuning, and inference ability training of large-scale operation and maintenance models.

[0034] It should be noted that the operation and maintenance of high-speed trains involves a large amount of specialized knowledge, including equipment structural principles, fault formation mechanisms, fault propagation patterns, maintenance process specifications, and historical maintenance experience. Since the general-purpose large-scale model is primarily pre-trained based on publicly available internet corpora, it lacks a background in rail transit expertise and struggles to accurately understand the specialized terminology, fault logic relationships, and maintenance decision-making rules within the high-speed train operation and maintenance scenario. Therefore, it is necessary to construct a dedicated domain corpus for rail transit operation and maintenance scenarios to provide a data foundation for subsequent knowledge distillation training and professional capability enhancement of the large-scale operation and maintenance model.

[0035] Step 4: Train the large-scale model in the operation and maintenance field based on the hierarchical fault knowledge distillation instruction fine-tuning mechanism to enable it to have the ability to understand faults, reason about causes, and generate maintenance decisions.

[0036] Specifically, such as Figure 3 As shown, this embodiment proposes a hierarchical fault knowledge distillation instruction fine-tuning mechanism, using a pre-trained large language model as the base model. A large-scale operation and maintenance model is constructed through a collaborative training method combining fault knowledge distillation and instruction fine-tuning. Specifically, fault phenomena, fault mechanisms, fault propagation paths, and maintenance and handling solutions are constructed as a unified fault reasoning chain, and domain knowledge is transferred to the parameter space of the large model through a multi-stage knowledge alignment method. First, fault reasoning samples are generated using the rail transit-specific domain corpus constructed in step 3, and the calculation formula is as follows: In the formula, This describes the fault symptoms. Indicates the fault formation mechanism, Indicates the fault propagation path, Indicate the repair and handling plan, This represents the number of training samples. Subsequently, by constructing a fault link mapping relationship and a fault link attention alignment mechanism, the model learns the causal relationship between fault states and strengthens the model's understanding of the train fault evolution process. At the same time, a unified training objective function is constructed by introducing loss functions of multiple dimensions, and its calculation formula is as follows: In the formula, Indicates the first A fault reasoning chain, This represents the fault chain encoding function, which primarily utilizes a text encoding model to map each node in the fault chain to a unified semantic space. It also employs a graph structure to organize fault entities and their relationships, enabling the model to learn the upstream and downstream dependencies between different fault nodes. Furthermore, and These represent the fault query vector and the key-value vector, respectively. Representing feature dimension, This is a fault association matrix that assigns weights λ to entity association edges based on association frequency, causal strength, and expert experience. Used for weighted enhancement of operational text features, the total loss function It is composed of losses from supervised fine-tuning, fault knowledge distillation, diagnostic path consistency, maintenance plan executability, and fault chain constraints. (Supervised fine-tuning loss) Used to ensure consistency between model-generated results and standard operational answers; fault knowledge distillation loss. This is used to effectively transfer expert diagnostic experience and domain knowledge to large-scale operation and maintenance models; diagnostic path consistency loss. This is used to ensure consistency between the model reasoning process and the expert fault analysis process, improving the logical reliability of fault cause analysis; and reducing the executability loss of repair solutions. Used to evaluate the feasibility and engineering applicability of solutions generated by the model based on historical maintenance success cases; fault chain constraint loss. It is used to maintain the causal relationship between fault phenomena, fault mechanisms, fault propagation paths, and maintenance and handling solutions, and to prevent the model from generating inference results that are inconsistent with the actual operation and maintenance logic. , , and This represents the corresponding weight parameter.

[0037] It is worth noting that, and A balanced cross-entropy loss function can be used. Consistency loss or absolute value loss can be used. L2 loss and cosine similarity loss can be used for collaborative supervision. , , and All are initialized to floating-point numbers between 0 and 1.

[0038] Step 5: Input the fault images collected on site into the multi-scale visual diagnostic architecture to obtain image recognition results containing fault category, fault location and fault evolution status information, and map them to a unified fault semantic space.

[0039] Specifically, fault images collected on-site are input into a multi-scale visual diagnostic architecture to obtain visual diagnostic results including fault category, fault location, fault level, and fault evolution stage. Subsequently, a fault semantic mapping module is used to associate and match the visual diagnostic results with standard fault terms, key component information, and maintenance knowledge in the rail transit operation and maintenance knowledge base, converting them into a unified fault semantic representation. This allows the image recognition results to be effectively connected with subsequent text reasoning processes, achieving a unified expression of visual information into operation and maintenance knowledge information. The fault semantic mapping module consists of a fault entity parsing unit, a knowledge association matching unit, and a fault profile generation unit. The fault entity parsing unit extracts fault category, fault location, and fault status information from the visual diagnostic results; the knowledge association matching unit establishes semantic associations between the visual diagnostic results and the rail transit operation and maintenance knowledge base; and the fault profile generation unit generates fault semantic descriptions in a unified format.

[0040] It should be noted that the output of the multi-scale visual diagnostic architecture is essentially structured diagnostic information at the visual perception level, mainly manifested as identification results such as fault category, fault location, fault level, and fault evolution state. However, the large-scale model reasoning process is mainly based on semantic information for knowledge understanding and logical reasoning, and cannot directly utilize visual diagnostic results to complete fault mechanism analysis and maintenance decision generation. Therefore, it is necessary to establish a semantic association mechanism between visual diagnostic results and the operation and maintenance knowledge system to realize the conversion of visual information into operation and maintenance knowledge expression.

[0041] It should also be noted that the same fault phenomenon may correspond to different risk levels and maintenance strategies under different operating environments, different equipment states, and different stages of fault development. Relying solely on a single fault label is insufficient to fully characterize the fault state. To improve the accuracy of subsequent large-scale model inference, this embodiment constructs a fault semantic mapping module. By fusing information on fault objects, fault characteristics, fault levels, evolution stages, and potential impacts, it generates a fault profile in a unified format, providing more complete semantic input for subsequent fault inference and maintenance decision generation.

[0042] Step 6: Input the fault semantic information, historical operation and maintenance log information, and user prompt words into the operation and maintenance domain big model to automatically generate corresponding fault handling solutions, maintenance processes, and spare parts replacement suggestions.

[0043] Specifically, the fault semantic information obtained in step 5 is fused with the historical operation and maintenance logs, historical fault records, and user-inputted operation and maintenance requirement prompts of the corresponding equipment, and then input into a large-scale operation and maintenance model that has been fine-tuned by hierarchical fault knowledge distillation instructions. This large-scale model combines the fault category, fault location, and fault evolution status from the fault semantic information, calls upon maintenance experience and handling records from historical operation and maintenance cases, and performs reasoning analysis on the causes of fault formation, potential impact range, and possible propagation paths. It also automatically generates targeted fault handling solutions, maintenance operation procedures, inspection item suggestions, and spare parts replacement suggestions based on the current operation and maintenance scenario, thereby achieving intelligent closed-loop support from fault identification to maintenance decision generation in the entire operation and maintenance scenario of high-speed trains.

[0044] It should be noted that the EMU (Electric Multiple Unit) operation and maintenance decision-making process not only relies on the current fault state, but is also closely related to the equipment's historical operating conditions, historical maintenance records, fault development patterns, and actual operation and maintenance needs. For the same fault phenomenon, the corresponding maintenance strategies may differ significantly depending on the equipment's operating stage, the degree of fault evolution, and the historical maintenance conditions. Therefore, it is difficult to generate an accurate, complete, and engineering-executable maintenance plan based solely on the current fault identification results. It is necessary to comprehensively integrate multi-source information such as fault semantic information, historical operation and maintenance log information, and user prompts, and use a large-scale model in the operation and maintenance domain for joint reasoning and decision generation.

[0045] This invention provides a system for generating operation and maintenance fault identification and handling solutions, used to implement the above method, including: The image acquisition module is used to acquire fault image data using fixed vision acquisition devices deployed in key maintenance areas of EMU trains; The benchmark dataset construction module is used to construct a benchmark dataset for the identification of faults in the entire operation and maintenance of high-speed trains based on the fault image data acquired by the image acquisition module. The multi-scale visual diagnosis module includes a local defect perception branch, a global state perception branch, a temporal change modeling branch, and a cross-branch dynamic coupling module. It is used to perform multi-branch collaborative diagnosis on the input fault image and output image recognition results containing fault category, fault location, and fault evolution state information. The corpus construction module is used to collect EMU operation and maintenance procedures, maintenance records, fault case database, expert diagnosis reports and historical operation and maintenance logs to build a corpus for the rail transit domain. The large-scale model in the operation and maintenance field is trained based on a hierarchical fault knowledge distillation instruction fine-tuning mechanism, and has the ability to understand faults, reason about causes, and generate maintenance decisions. The fault semantic mapping module is used to map the image recognition results output by the multi-scale visual diagnosis module to a unified fault semantic space to form fault semantic information. The solution generation module is used to input fault semantic information, historical operation and maintenance log information of the corresponding equipment, and user prompts into the operation and maintenance domain big model, and automatically generate corresponding fault handling solutions, maintenance processes, and spare parts replacement suggestions.

[0046] An electronic device includes: one or more processors; and a storage device for storing one or more programs; when the programs are executed by the one or more processors, the processors cause the processors to perform the methods described above.

[0047] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0048] The technical solution of this embodiment achieves accurate identification and status diagnosis of fault characteristics of key components of EMU by constructing a multi-scale visual diagnostic architecture that integrates local defect perception, global state perception, and temporal change modeling branches. By constructing a corpus for the rail transit domain and adopting a hierarchical fault knowledge distillation instruction fine-tuning mechanism, it achieves efficient transfer of operation and maintenance knowledge to a large model and enhances professional reasoning capabilities. Furthermore, by combining a fault semantic mapping module to complete the semantic mapping between visual diagnostic results and the operation and maintenance knowledge system, and by integrating historical operation and maintenance log information and user prompts, it automatically generates fault handling plans, maintenance procedures, and spare parts replacement suggestions. This realizes intelligent closed-loop processing of fault identification, fault analysis, and maintenance decision generation in the entire operation and maintenance scenario of EMUs, effectively improving the accuracy of fault diagnosis, the scientific nature of maintenance decisions, and the level of intelligent operation and maintenance management.

[0049] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for generating operation and maintenance fault identification and handling solutions, characterized in that, Includes the following steps: Step 1: Acquire fault image data using fixed visual acquisition devices deployed in key operation and maintenance areas of EMU trains, and construct a benchmark dataset for EMU full-domain operation and maintenance fault identification. Step 2: Construct a multi-scale visual diagnostic architecture based on the collaborative fusion of local defect perception, global state perception, and temporal change modeling branches, and train, optimize, and update the parameters of the benchmark dataset; Step 3: Collect EMU operation and maintenance procedures, maintenance records, fault case database, expert diagnosis reports, and historical operation and maintenance logs to build a corpus for the rail transit specific domain; Step 4: Train the large model in the operation and maintenance field based on the hierarchical fault knowledge distillation instruction fine-tuning mechanism to enable it to have the ability to understand faults, reason about causes, and generate maintenance decisions. Step 5: Input the fault images collected on site into the multi-scale visual diagnostic architecture to obtain image recognition results containing fault category, fault location and fault evolution status information, and map them to a unified fault semantic space. Step 6: Integrate the fault semantic information obtained in Step 5 with the historical operation and maintenance logs, historical fault records, and operation and maintenance requirement prompts entered by the user for the corresponding equipment, and input them together into the operation and maintenance domain big model trained in Step 4 to automatically generate corresponding fault handling solutions, maintenance processes, and spare parts replacement suggestions.

2. The method for generating operation and maintenance fault identification and handling solutions as described in claim 1, characterized in that, Step 1 is achieved in the following way: Step 1.1: Use a fixed industrial camera to continuously acquire images of the EMU bogie, pantograph, door system, braking system, and car body structure. Step 1.2: Extract keyframes from the video stream every 20 frames at a preset time interval; Step 1.3: Construct a fault label set by using intelligent assisted labeling as the main method and manual labeling, verification, and calibration as a supplementary method; Step 1.4: Establish a three-dimensional annotation system that includes fault category labels, fault area labels, and fault level labels; Step 1.5: Construct the training set, validation set, and test set in a ratio of 8:1:

2.

3. The method for generating operation and maintenance fault identification and handling solutions as described in claim 1, characterized in that, The calculation formulas for the local defect perception branch, global state perception branch, and temporal change modeling branch in step 2 are as follows: In the formula, For input image, This represents operation based on channel connection. The window size is Deformable convolution, It is a deformable convolution with a window size of 5×5. It is a deformable convolution with a window size of 7×7; q and k Represents query and key, Input image generated through a linear layer A copy, Represents matrix multiplication. for q 1. k 1 and q 2. k 2. Calculate the difference after matrix multiplication; It is a gated structure consisting of several convolutional layers, normalization layers, and activation layers, with the last layer using the GeLU activation function. Represents element-wise multiplication; This represents a selective scanning mechanism path controlled by variance; , and These refer to the results of the local defect perception branch, the global state perception branch, and the temporal change modeling branch, respectively.

4. The method for generating operation and maintenance fault identification and handling solutions as described in claim 3, characterized in that, The branch collaborative fusion in step 2 is , and Information fusion is achieved through a cross-branch dynamic coupling module, and its calculation formula is as follows: In the formula, , and These represent trainable parameters, initialized to 0.5, 0.25, and 0.25 respectively. This represents the result of fusing information from different branches.

5. The method for generating operation and maintenance fault identification and handling solutions as described in claim 1, characterized in that, Step 3 is achieved in the following way: Step 3.1: Collect the EMU operation and maintenance procedures, maintenance operation instructions, equipment technical manuals, fault case library, historical maintenance work orders, expert diagnosis reports and historical operation and maintenance logs to form the original operation and maintenance text data pool. Step 3.2: Perform format unification processing on the original operation and maintenance text data pool to eliminate format differences between data from different sources, and complete text cleaning, duplicate content removal, garbled character correction, and invalid field filtering. Step 3.3: Establish domain entity recognition rules based on the rail transit professional terminology dictionary, and automatically extract fault phenomena, faulty components, fault causes, maintenance measures, spare parts names, detection parameters and operating environment information from the text to form a standardized set of operation and maintenance entities; Step 3.4: Establish entity associations based on the operation and maintenance business logic, map the fault phenomena, fault causes, faulty components and maintenance measures, construct a fault knowledge link, and realize the complete association from fault manifestation to maintenance and handling process; Step 3.5: Perform time series correlation analysis on the operation and maintenance records generated at different operating stages for the same fault, extract information on the entire process of fault occurrence, fault expansion, fault deterioration and fault handling, and form a fault evolution knowledge sample; Step 3.6: Construct a knowledge graph for rail transit operation and maintenance using fault knowledge links and fault evolution knowledge samples, and organize and manage key components of EMU trains, typical fault modes, fault propagation paths and maintenance strategies in a unified manner; Step 3.7: Automatically generate fault description question-and-answer pairs, fault cause analysis question-and-answer pairs, maintenance process question-and-answer pairs, and maintenance decision question-and-answer pairs based on the knowledge graph to form a multi-level operation and maintenance instruction dataset; Step 3.8: Combining the structured operation and maintenance knowledge, fault evolution knowledge, and operation and maintenance instruction dataset obtained in steps 3.3 to 3.7, construct a corpus for the rail transit domain, providing basic data support for subsequent knowledge injection, instruction fine-tuning, and inference ability training of large-scale operation and maintenance models.

6. The method for generating an operation and maintenance fault identification and handling solution as described in claim 1, characterized in that, Step 4 trains the large model in the operation and maintenance domain based on the hierarchical fault knowledge distillation instruction fine-tuning mechanism. This involves constructing a unified fault reasoning chain from fault phenomena, fault mechanisms, fault propagation paths, and maintenance and handling solutions, and transferring domain knowledge to the parameter space of the large model through a multi-stage knowledge alignment method.

7. The method for generating an operation and maintenance fault identification and handling plan as described in claim 6, characterized in that, The construction of the fault reasoning chain and the transfer of domain knowledge to the large model parameter space are achieved through the following methods: First, fault reasoning samples are generated using the rail transit-specific domain corpus constructed in step 3. The calculation formula is as follows: In the formula, This describes the fault symptoms. Indicates the fault formation mechanism, Indicates the fault propagation path, Indicate the repair and handling plan, Indicates the number of training samples; Subsequently, by constructing a fault link mapping relationship and a fault link attention alignment mechanism, the model learns the causal relationship between fault states and strengthens the model's understanding of the train fault evolution process. At the same time, a unified training objective function is constructed by introducing loss functions of multiple dimensions, and its calculation formula is as follows: In the formula, Indicates the first A fault reasoning chain, This represents the fault chain coding function. and These represent the fault query vector and the key-value vector, respectively. Representing feature dimension, This is a fault association matrix that assigns weights λ to entity association edges based on association frequency, causal strength, and expert experience. Used to weight and enhance the features of operation and maintenance text. For the total loss function, To monitor and fine-tune losses, For fault knowledge distillation loss, To diagnose path consistency loss, Loss due to the feasibility of the repair plan. For failure chain constraint loss, , , and This represents the corresponding weight parameter.

8. The method for generating operation and maintenance fault identification and handling solutions as described in claim 7, characterized in that, The and Use the balanced cross-entropy loss function Use consistency loss or absolute value loss. L2 loss and cosine similarity loss are used for collaborative supervision. , , and All are initialized to floating-point numbers between 0 and 1.

9. The method for generating an operation and maintenance fault identification and handling solution as described in claim 1, characterized in that, Step 5, mapping to a unified fault semantic space, is achieved in the following way: the visual diagnostic results are associated and matched with standard fault terms, key component information and maintenance knowledge in the rail transit operation and maintenance knowledge base using the fault semantic mapping module, and converted into unified fault semantic information, so that the image recognition results can be effectively connected with the subsequent text reasoning process.

10. The method for generating an operation and maintenance fault identification and handling plan as described in claim 9, characterized in that, Step 6, which automatically generates corresponding fault handling solutions, maintenance procedures, and spare parts replacement suggestions, is achieved by: combining the fault category, fault location, and fault evolution status in the fault semantic information of the large-scale operation and maintenance model, calling on maintenance experience and handling records in historical operation and maintenance cases, reasoning and analyzing the causes of fault formation, potential impact range, and possible propagation paths, and automatically generating targeted fault handling solutions, maintenance operation procedures, inspection item suggestions, and spare parts replacement suggestions in combination with the current operation and maintenance scenario.

11. A system for generating operation and maintenance fault identification and handling solutions, used to implement the method described in any one of claims 1 to 10, characterized in that, include: The image acquisition module is used to acquire fault image data using fixed vision acquisition devices deployed in key maintenance areas of EMU trains; The benchmark dataset construction module is used to construct a benchmark dataset for the identification of faults in the entire operation and maintenance of high-speed trains based on the fault image data acquired by the image acquisition module. The multi-scale visual diagnosis module includes a local defect perception branch, a global state perception branch, a temporal change modeling branch, and a cross-branch dynamic coupling module. It is used to perform multi-branch collaborative diagnosis on the input fault image and output image recognition results containing fault category, fault location, and fault evolution state information. The corpus construction module is used to collect EMU operation and maintenance procedures, maintenance records, fault case database, expert diagnosis reports and historical operation and maintenance logs to build a corpus for the rail transit domain. The large-scale model in the operation and maintenance field is trained based on a hierarchical fault knowledge distillation instruction fine-tuning mechanism, and has the ability to understand faults, reason about causes, and generate maintenance decisions. The fault semantic mapping module is used to map the image recognition results output by the multi-scale visual diagnosis module to a unified fault semantic space to form fault semantic information. The solution generation module is used to input fault semantic information, historical operation and maintenance log information of the corresponding equipment, and user prompts into the operation and maintenance domain big model, and automatically generate corresponding fault handling solutions, maintenance processes, and spare parts replacement suggestions.

12. The maintenance fault identification and handling solution generation system as described in claim 11, characterized in that, The fault semantic mapping module includes: The fault entity parsing unit is used to extract fault category, fault location and fault status information from the visual diagnostic results; The knowledge association matching unit is used to establish semantic associations between visual diagnostic results and the rail transit operation and maintenance knowledge base; The fault profile generation unit is used to generate fault semantic description results in a unified format.

13. An electronic device, characterized in that, include: One or more processors; as well as Storage device for storing one or more programs; When the program is executed by the one or more processors, the processors implement the method as described in any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the method as described in any one of claims 1 to 10.

Images