Locomotive maintenance data standardization method and device, electronic equipment and storage medium
By combining keyword matching and deduplication filtering with automatic annotation of the smallest maintenance unit files, and by training with PaddlePaddle deep learning and large language models, the standardization problem of locomotive maintenance data was solved, achieving data standardization and automation, and reducing the error rate.
Patent Information
- Application Number
- CN202511125017.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-12
- Publication Date
- 2025-11-07
AI Technical Summary
Existing locomotive maintenance data suffers from problems such as insufficient data quantity, non-standard format, difficulty in summarizing fault types, and high data processing costs. The low level of automation leads to standardization challenges due to non-standard data formats, high manual processing costs, and low level of automated analysis.
Data cleaning is performed using keyword matching and deduplication filtering, automatic annotation is performed using the smallest repair unit file, structured pattern hints and extraction are performed using PaddlePaddle deep learning, and data expansion, clustering and transformation are achieved through training with a large language model, thereby reducing the standardization error rate.
It has achieved the standardization, normalization, and automation of locomotive maintenance data, reduced the standardization error rate, and improved the automation and accuracy of data processing.
Smart Images

Figure CN120910585A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of natural language processing, and particularly relates to a locomotive maintenance data standardization method and device, an electronic equipment and a storage medium. BACKGROUND
[0002] At present, the existing maintenance data is manually entered by each maintenance team, and there are four problems of lacking data quantity, non-standard data format, difficult fault type summary and high data processing cost. The traditional manual data cleaning and manual annotation have low automation degree, and the maintenance data standardization is faced with problems of non-standard data format, high manual processing cost, low automatic analysis degree and manual analysis failure. SUMMARY
[0003] The main purpose of the embodiment of the application is to provide a locomotive maintenance data standardization method and device, an electronic equipment and a storage medium, which realize locomotive maintenance data standardization and automation, and reduce the standardization failure rate.
[0004] One aspect of the application provides a locomotive maintenance data standardization method, comprising: obtaining an original data set, performing data cleaning and annotation on the original data set to obtain a first locomotive maintenance data set; performing data expansion, data clustering and data conversion processing on the first locomotive maintenance data set to obtain a second locomotive maintenance data set; performing structured mode prompting and structured extraction on the second locomotive maintenance data set through PaddlePaddle deep learning to obtain an extraction result; training through the extraction result to obtain a standardized large language model; performing data standardization processing on target locomotive maintenance data through the standardized large language model to obtain a data standardization result.
[0005] According to the locomotive maintenance data standardization method, wherein the original data set is cleaned and annotated to obtain the first locomotive maintenance data set, comprising: cleaning the original data set by keyword matching and de-duplication screening; obtaining a minimum maintenance unit file of the maintenance record in the original data set, and supplementing and automatically annotating the original data set according to the minimum maintenance unit file and the keyword to obtain the first locomotive maintenance data set.
[0006] According to the locomotive maintenance data standardization method, wherein the first locomotive maintenance data set is obtained, and expansion, clustering and conversion processing are performed on the first locomotive maintenance data set to obtain the second locomotive maintenance data set, comprising: The expansion processing includes label expansion and data combination of the first locomotive maintenance data set after cleaning and labeling by using a python script. The clustering processing includes fault type balancing processing of the first locomotive maintenance data set according to the fault type proportion in the first locomotive maintenance data set. The conversion processing includes converting the first locomotive maintenance data set into a JSON format for model training.
[0007] According to the locomotive maintenance data standardization method, the structured mode prompting of the second locomotive maintenance data set by PaddlePaddle deep learning includes: The spotname word symbol segment, the assoname word symbol segment, and the special symbol in the structured mode prompting of the PaddlePaddle deep learning are processed, including name positioning of the target data of the extraction task by the spotname word symbol segment, name association of the target data by the assoname word symbol segment, and adding the special symbol to the spotname word symbol segment, the assoname word symbol segment, and the input text sequence of the second locomotive maintenance data set, so that the second locomotive maintenance data set shares semantic knowledge of labels and tasks.
[0008] According to the locomotive maintenance data standardization method, the structured extraction of the second locomotive maintenance data set by PaddlePaddle deep learning includes: The spotname word symbol segment of the structured mode prompting of the PaddlePaddle deep learning determines the specific information segment of the positioning type of the second locomotive maintenance data set, the assoname word symbol segment determines the specific information segment of the second locomotive maintenance data set and the associated information of the positioning information, and the infoname word symbol segment determines the text span of the information block positioned or associated by the second locomotive maintenance data set. The hidden representation of the spotname word symbol segment, the assoname word symbol segment, and the infoname word symbol segment is calculated by using the Transformer encoder. According to the hidden representation, the conditional probability of the word symbol segment is predicted by the Transformer decoder, the linear SEL expression of the second locomotive maintenance data set is determined according to the conditional probability, the linear SEL expression is converted, and the extraction result is obtained.
[0009] According to the locomotive maintenance data standardization method, the standardization large language model is obtained by training the extraction result, including: The extraction result is divided into a training set, a validation set, and a test set according to a preset proportion. The weight parameter initialization of performing network training is performed, the extraction result is batch input, and forward transmission processing is performed; Based on the binary classification cross entropy loss function, the network weight parameters are optimized by using the AdamW algorithm until the preset step number of training is completed, and the standardized large language model is obtained.
[0010] According to the locomotive maintenance data standardization method, the method further comprises: The performance of the standardized large language model is detected by using precision, recall rate and F1 score, and the performance of the standardized large language model is visualized according to the performance detection, the batch data input, the forward transmission and the information extraction of the standardized large language model.
[0011] Another aspect of the embodiment of the application provides a locomotive maintenance data standardization device, comprising: The first module is used for obtaining an original data set, performing data cleaning and labeling on the original data set, and obtaining a first locomotive maintenance data set; The second module is used for performing data expansion, data clustering and data conversion processing on the first locomotive maintenance data set, and obtaining a second locomotive maintenance data set; The third module is used for performing structured mode prompting and structured extraction on the second locomotive maintenance data set through PaddlePaddle deep learning, and obtaining an extraction result; The fourth module is used for training through the extraction result, and obtaining a standardized large language model; The fifth module is used for performing data standardization processing on target locomotive maintenance data through the standardized large language model, and obtaining a data standardization result.
[0012] Another aspect of the embodiment of the application provides an electronic device comprising a processor and a memory. The memory is used for storing a program. The processor executes the program to realize the method as described above.
[0013] The embodiment of the application further discloses a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method as described above.
[0014] The beneficial effects of the present application are: through automatic data cleaning by keyword matching and deduplication screening, then combined with the minimum repair unit file of the maintenance record, automatic labeling is realized by using keyword (minimum repair unit) matching, realizing the automation of locomotive maintenance data cleaning and labeling processing; the scale of the maintenance data is expanded by using the script mode, and the problem type is balanced by using the data clustering method, which automatically expands the scale of the maintenance data and automatically balances the data categories; based on the artificial recording of locomotive maintenance data, a large language model is introduced to perform the information extraction task of the locomotive maintenance data, through inputting the maintenance record, the component name, the accessory number, the component primary number, the primary accessory name, the component secondary name, the fault label, etc. of the maintenance data after automatic standardization can be obtained, and the standardization, automation and low error rate of the locomotive maintenance data are realized. BRIEF DESCRIPTION OF DRAWINGS
[0015] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which: Figure 1 is a locomotive maintenance data standardization method flow diagram of an embodiment of the present application.
[0016] Figure 2 is a standardization test result example diagram based on locomotive maintenance data using a large language model of an embodiment of the present application.
[0017] Figure 3 is a locomotive maintenance data standardization device diagram of an embodiment of the present application. DETAILED DESCRIPTION
[0018] Embodiments of the present application will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. In the following description, the suffixes used to denote elements such as "module", "component", or "unit" are used only for the convenience of the description of the present application, and have no particular meaning in themselves. Therefore, "module", "component", or "unit" can be used mixedly. "First", "second", and the like are only used to distinguish technical features for the purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the chronological relationship of indicated technical features. In the following description, the consecutive numbers of the method steps are for the convenience of review and understanding, and adjusting the implementation order between the steps will not affect the technical effects achieved by the technical solution of the present application, in combination with the overall technical solution of the present application and the logical relationship between the steps. The embodiments described below by reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.
[0019] REFERENCE Figure 1 ,Figure 1 This is a flowchart illustrating the standardization method for locomotive maintenance data, which includes, but is not limited to, steps S100~S500: S100: Obtain the original dataset, perform data cleaning and labeling on the original dataset to obtain the first locomotive maintenance dataset.
[0020] In some embodiments, the original dataset is cleaned by keyword matching and deduplication filtering; the minimum maintenance unit file of the maintenance records in the original dataset is obtained, and the original dataset is supplemented with text and automatically labeled according to the minimum maintenance unit file and keywords to obtain the first locomotive maintenance dataset.
[0021] In some embodiments, such as Figure 2 The locomotive maintenance data shown includes maintenance data for 11 subsystems, including LKJ, safety equipment, auxiliary units, high-voltage electrical equipment, train tail, traction converter, traction motor, general electrical equipment, braking system, main transformer, bogie and car body.
[0022] In some embodiments, the dataset used for training the standardized large language model of locomotive maintenance data is maintenance data that includes fault replacements and preventative replacements, selected from the pre-fault maintenance data of overhauled locomotives from 2017 to 2023. The pre-fault maintenance dataset of overhauled locomotives is collected from the field, and combined with maintenance specification documents such as the minimum repair unit, data is cleaned and filtered through keyword matching, and data is labeled using text supplementation to complete the labeling of parts, accessories, fault types, etc. The dataset is saved as an Excel spreadsheet file.
[0023] S200 performs data augmentation, data clustering, and data transformation on the first locomotive maintenance dataset to obtain the second locomotive maintenance dataset.
[0024] In some embodiments, data augmentation automates the cleaning and labeling of video security system fault files from 2020 to 2023, then performs label augmentation and data combination, increasing the amount of locomotive maintenance data. The data clustering function balances the proportion of each fault type through clustering methods, ensuring a balanced distribution of fault types in the dataset, which is beneficial for subsequent model-based balanced extraction of fault features. The data conversion function converts Excel data into JSON data format for model training, reducing the time spent on manual labeling.
[0025] The augmentation process includes using Python scripts to augment labels and combine data from the cleaned and labeled first locomotive maintenance dataset.
[0026] In some embodiments, the data of the original data set is obtained, a new data set is formed, and is named as an expanded data set. Data clustering is to first combine the fault description and treatment method of each record into a text, and then use cosine similarity greater than a threshold value (the threshold value of the present application is set to 0.8) to cluster as a class. After that, in order to balance the classes in the data set, the number of clusters is combined to extract each class of cluster file by proportion (the proportion of the present application is set to 10%) and rounding up. Finally, 1429 data is extracted and combined with unclustered data to form a new data set, named as a clustered data set. Data conversion is to convert the original data set, the expanded data set and the clustered data set into a data set format suitable for model training. Using python, according to the record of component name, accessory number, component first-level number, first-level accessory name, component second-level name, fault label in the position of fault description and treatment method, position matching and text supplement are performed, data labeling is completed, and excel table file is converted into json format file suitable for model training.
[0027] S300, performing structured pattern prompting and structured extraction on the second locomotive maintenance data set by PaddlePaddle deep learning to obtain an extraction result.
[0028] In some embodiments, the spotname word symbol segment, the assoname word symbol segment and the special symbol of the structured pattern prompting of PaddlePaddle deep learning are processed, including positioning the name of the target data by the spotname word symbol segment, associating the name of the target data by the assoname word symbol segment, and adding the special symbol to the spotname word symbol segment, the assoname word symbol segment and the input text sequence of the second locomotive maintenance data set, so that the second locomotive maintenance data set shares semantic knowledge of labels and tasks.
[0029] In some embodiments, the spotname word symbol segment of the structured pattern prompting of PaddlePaddle deep learning determines the specific information segment of the positioning type of the second locomotive maintenance data set, the assoname word symbol segment determines the specific information segment of the second locomotive maintenance data set having associated information with the positioning information, and the infoname word symbol segment determines the text span of the information block positioned or associated by the second locomotive maintenance data set. The hidden representation of the spotname word symbol segment, the assoname word symbol segment and the infoname word symbol segment is calculated by using the Transformer encoder. According to the hidden representation, the conditional probability of the word symbol segment is predicted by using the Transformer decoder. The linear SEL expression of the second locomotive maintenance data set is determined according to the conditional probability. The linear SEL expression is converted to obtain the extraction result.
[0030] In some embodiments, the PaddlePaddle deep learning framework can also use the pyTorch, Tensorflow or cafffe framework.
[0031] S400, training is performed through the extraction result to obtain a standardized large language model.
[0032] In some embodiments, the extraction result is divided into a training set, a validation set and a test set according to a preset ratio; the weight parameter initialization of network training is performed, the extraction result is input in batches and forward propagation processing is performed; based on a binary classification cross-entropy loss function, the network weight parameter is optimized by using an AdamW algorithm until the training of a preset number of steps is completed, and a standardized large language model is obtained.
[0033] In some embodiments, batch text data is input step by step, and the large language model is forward propagated. Guided by a binary classification cross-entropy loss function, the network weight parameter is optimized by using an AdamW algorithm. The model is verified by setting a verification step number. The loss function of the validation set is used as the model saving criterion, and the model with the highest validation set accuracy is saved. After the set training step number is reached, the iteration training is stopped, and the three different data sets obtained are used for model training. The ratio of the data set during training is set to 1:1:8. The large language model uses a UIE large model. This model is a general information extraction framework released by Baidu, which has the advantages of simple use, cost reduction and efficiency improvement, and fast adaptation to specific targets. The learning rate is 0.00001, the batch size is 8, the number of GPUs is 8, and the epoch is 100. During training, the best neural network weight is saved according to the validation set accuracy. Table 1 shows the details.
[0034] Table 1 Standardization training result of large language model based on locomotive maintenance data
[0035] In some embodiments, the locomotive maintenance data standardized large language model is UIE, and other large language models such as LLAMA can also be used.
[0036] S500, data standardization processing is performed on the target locomotive maintenance data by using the standardized large language model, and a data standardization result is obtained.
[0037] In some embodiments, the locomotive maintenance data standardized large language model test function includes text batch data input, large language model forward propagation and information extraction performance display function. Batch text test data is input, the large language model forward propagation result is calculated, and finally the information extraction result of the maintenance record is visualized and displayed.
[0038] The trained three locomotive repair data standardization large language models are input into the trained locomotive repair data standardization large language models one by one corresponding to the divided test data set, and the performance of each data set in the locomotive repair data standardization large language model is obtained, as shown in Table 2 below.
[0039] Table 2 Standardization performance evaluation comparison table
[0040] In some embodiments, the locomotive repair data standardization large language model performance analysis module uses precision, recall, F1 score and other performance indicators to evaluate the performance of the large language model in the information extraction task of different data sets.
[0041] Figure 3 is a locomotive repair data standardization analysis device diagram of an embodiment of the present application. The device includes a first module 310, a second module 320, a third module 330, a fourth module 340 and a fifth module 350.
[0042] The first module is configured to obtain an original data set, clean and label the original data set, and obtain a first locomotive repair data set. The second module is configured to perform data augmentation, data clustering and data conversion processing on the first locomotive repair data set, and obtain a second locomotive repair data set. The third module is configured to perform structured mode prompting and structured extraction on the second locomotive repair data set through PaddlePaddle deep learning, and obtain an extraction result. The fourth module is configured to train through the extraction result, and obtain a standardization large language model. The fifth module is configured to perform data standardization processing on target locomotive repair data through the standardization large language model, and obtain a data standardization result.
[0043] Exemplarily, under cooperation of the first module to the fifth module in the device, the embodiment device can implement any one of the aforementioned locomotive maintenance data standardization methods, that is, obtaining an original data set, performing data cleaning and labeling on the original data set to obtain a first locomotive maintenance data set; performing data expansion, data clustering and data conversion processing on the first locomotive maintenance data set to obtain a second locomotive maintenance data set; performing structured mode prompting and structured extraction on the second locomotive maintenance data set through PaddlePaddle deep learning to obtain an extraction result; training through the extraction result to obtain a standardized large language model; and performing data standardization processing on target locomotive maintenance data through the standardized large language model to obtain a data standardization result. The beneficial effects of the present application are as follows: through automatic data cleaning by keyword matching and deduplication screening, and then combining with a minimum maintenance unit file of a maintenance record, automatic annotation is performed by keyword (minimum maintenance unit) matching, locomotive maintenance data cleaning and labeling processing automation is realized; the scale of the maintenance data is expanded in a script mode, and the data clustering method is used to balance the problem types, so as to automatically expand the scale of the maintenance data and automatically balance the data categories; based on the artificial record locomotive maintenance data, a large language model is introduced to perform an information extraction task of the locomotive maintenance data, through input of the maintenance record, the component name, the accessory number, the component primary number, the primary accessory name, the component secondary name, the fault label and the like of the maintenance data after automatic standardization can be obtained, and locomotive maintenance data standardization, automation and low error rate are realized.
[0044] The embodiment of the present application further provides an electronic device, which comprises a processor and a memory; The memory stores a program; The processor executes the program to perform the aforementioned locomotive maintenance data standardization method; the electronic device has the function of carrying and running the software system for locomotive maintenance data standardization provided by the embodiment of the present application, for example, a personal computer, a mini computer, a main frame, a workstation, a network or a distributed computing environment, a separate or integrated computer platform, or communication with a charged particle tool or other imaging device, and the like.
[0045] The embodiment of the present application further provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the locomotive maintenance data standardization method as described above.
[0046] In some alternative embodiments, the function / operations recited in the block diagrams can occur in different orders than the orders described in the operational illustrations. For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality / operations involved. Also, although the embodiments presented in the flow diagrams have been presented in serial fashion, it should be understood that the disclosure is not limited to the order of the blocks, as some blocks can, in fact, occur in a different, and potentially simultaneous, order. In addition, the embodiments presented in the flow diagrams are provided by way of example only and are not exhaustive of implementing the methods presented. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of a larger operation are independently executed.
[0047] The embodiments of the present application further disclose a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the aforementioned locomotive repair data standardization method.
[0048] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the described functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be appreciated that detailed discussion of the actual implementation of each module is unnecessary to an understanding of the present application. Rather, the actual implementation of the modules, in light of the attributes, functions and internal relationships of the various functional modules disclosed herein, will be apparent to one of ordinary skill in the art in view of the present disclosure. Thus, the present application is not limited to the embodiments described herein which can be amended or modified by one of ordinary skill in the art without departing from the scope of the present application as set forth in the appended claims. Furthermore, the disclosed particular concepts are merely illustrative and are not intended to limit the scope of the present application, which is defined by the full scope of the appended claims and equivalents thereof.
[0049] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions of the present application can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.
[0050] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, which can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus or device, such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute the instructions, or in conjunction with these instruction execution systems, apparatus or devices. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by or in connection with an instruction execution system, apparatus or device, or in conjunction with these instruction execution systems, apparatus or devices.
[0051] More specific examples (non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CD ROM). In addition, the computer readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optical scanning of the paper or other medium, followed by editing, interpreting or otherwise processing, if necessary, in other suitable ways, to be electronically obtained and then stored in the computer memory.
[0052] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware which is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, can be used: a combination of discrete logic circuits having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having logic gates, field programmable gate arrays (FPGA), or others.
[0053] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples.
[0054] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, alternatives and variations can be made thereto without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.
[0055] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are included in the scope defined by the claims of the present application.
Claims
1. A method for locomotive service data standardization, the method comprising: The method comprises the following steps: obtaining an original data set, performing data cleaning and labeling on the original data set to obtain a first locomotive maintenance data set; performing data expansion, data clustering and data conversion processing on the first locomotive maintenance data set to obtain a second locomotive maintenance data set; performing structured mode prompting and structured extraction on the second locomotive maintenance data set through PaddlePaddle deep learning to obtain an extraction result; training through the extraction result to obtain a standardized large language model; performing data standardization processing on target locomotive maintenance data through the standardized large language model to obtain a data standardization result.
2. The locomotive service data standardization method of claim 1, wherein, The method comprises the following steps: performing data cleaning on the original data set by keyword matching and de-duplication screening; obtaining a minimum maintenance unit file of maintenance records in the original data set, and performing text supplement and automatic labeling on the original data set according to the minimum maintenance unit file and the keyword to obtain the first locomotive maintenance data set.
3. The locomotive service data standardization method of claim 1, wherein, The method comprises the following steps: The expansion processing comprises label expansion and data combination of the first locomotive maintenance data set cleaned and labeled through a python script. The clustering processing comprises fault type balancing processing of the first locomotive maintenance data set according to the fault type proportion in the first locomotive maintenance data set. The conversion processing comprises converting the first locomotive maintenance data set into a JSON format for model training.
4. The locomotive service data standardization method of claim 1, wherein, The method comprises the following steps: The spotname word symbol segment, the assoname word symbol segment and the special symbol of the structured mode prompting of the PaddlePaddle deep learning are processed, which comprises name positioning of the target data of the extraction task through the spotname word symbol segment, name association of the target data through the assoname word symbol segment, and adding the special symbol to the spotname word symbol segment, the assoname word symbol segment and the input text sequence of the second locomotive maintenance data set, so that the second locomotive maintenance data set shares semantic knowledge of labels and tasks.
5. The locomotive service data standardization method of claim 4, wherein, The method comprises the following steps: The spotname word symbol segment of the structured mode prompting of the PaddlePaddle deep learning is used to determine the specific information segment of the positioning type of the second locomotive maintenance data set, the assoname word symbol segment is used to determine the specific information segment of the second locomotive maintenance data set having associated information with the positioning information, and the infoname word symbol segment is used to determine the text span of the information block positioned or associated by the second locomotive maintenance data set; The Transformer encoder is used to calculate the hidden representation of the spotname word symbol segment, the assoname word symbol segment and the infoname word symbol segment; According to the hidden representation, a conditional probability of a word symbol segment is predicted by a Transformer decoder, a linear SEL expression of the second locomotive maintenance data set is determined according to the conditional probability, the linear SEL expression is converted to obtain an extraction result.
6. The locomotive service data standardization method of claim 1, wherein, The training by the extraction result obtains a standardized large language model, which includes: The extraction result is divided into a training set, a validation set and a test set according to a preset proportion; The weight parameter initialization of network training is performed, the extraction result is input in batches and forward propagation processing is performed; Based on the binary classification cross-entropy loss function, the network weight parameters are optimized by using the AdamW algorithm until the training of the preset step number is completed, and the standardized large language model is obtained.
7. The locomotive service data standardization method of claim 6, wherein, The method further includes: The performance of the standardized large language model is detected by using precision, recall and F1 score, and performance visualization display is performed on the performance of the standardized large language model in batch data input, forward propagation and information extraction.
8. A locomotive service data standardization apparatus, characterized by, It includes: A first module is configured to obtain an original data set, clean and label the original data set, and obtain a first locomotive maintenance data set; A second module is configured to perform data expansion, data clustering and data conversion processing on the first locomotive maintenance data set to obtain a second locomotive maintenance data set; A third module is configured to perform structured mode prompting and structured extraction on the second locomotive maintenance data set by PaddlePaddle deep learning to obtain an extraction result; A fourth module is configured to train by the extraction result to obtain a standardized large language model; A fifth module is configured to perform data standardization processing on target locomotive maintenance data by the standardized large language model to obtain a data standardization result.
9. An electronic device, comprising: It includes a processor and a memory; The memory is used to store programs; The processor executes the programs to realize the locomotive maintenance data standardization method in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The storage medium stores programs, and the programs are executed by the processor to realize the locomotive maintenance data standardization method in any one of claims 1-7.
Citation Information
Cited By
Data management method, electronic equipment and computer readable storage medium
CN121455940A
