Batch classification method and device for railway scientific and technological documents

By combining the historical batch feature extraction module and graph encoder, the feature extraction and classification of railway science and technology literature is optimized, and the problem of unstable classification in the existing system is solved, more accurate classification of railway science and technology literature is achieved, and user experience is improved.

CN118568600BActive Publication Date: 2024-11-05INST OF SCI & TECH INFORMATION OF CHINA ACAD OF RAILWAY SCI GRP CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411046773.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-05
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

Due to the strong cross-section of industry in the industry and the existing railway science and technology literature classification system, it is difficult to achieve comprehensive identification of manual labeling, and the classification effect is unstable, which affects efficiency and user experience.

Method used

The first and second railway science and technology literature feature extraction modules of historical batches are used to generate samples in combination with the graph encoder, and the feature extraction module parameters are optimized through extra-domain scoring and fusion scores, and the pre-trained Chinese BERT model and linear classifier are used for classification.

Benefits of technology

A more accurate and complete classification of railway scientific and technological documents has been achieved, and the classification effect and user experience have been improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118568600B_ABST
    Figure CN118568600B_ABST
Patent Text Reader

Abstract

The present application provides a method and device for batch classification of railway scientific and technological documents, and relates to the technical field of railway scientific and technological document classification. The method includes: determining the railway scientific and technological documents that belong to the in-domain documents in the initial documents of the current batch to be classified based on the fused out-of-domain scores; optimizing the parameters in the first railway scientific and technological document feature extraction module of the historical batch for each of the railway scientific and technological documents, and optimizing the parameters of the second railway scientific and technological document feature extraction module of the historical batch based on the historical parameters of the first railway scientific and technological document feature extraction module; classifying the current batch of railway scientific and technological documents based on the first railway scientific and technological document feature extraction module and classifier of the current batch. Compared with the traditional scientific and technological document classification system, it will be able to obtain more complete classification information more accurately, improve the effect of railway scientific and technological document classification and user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of railway scientific and technological literature classification, and in particular, to a method and device for batch classification of railway scientific and technological literature. Background Art

[0002] At present, the railway science and technology literature classification system mainly adopts a general literature classification method, for example, first perform manual annotation, train the model based on the manually annotated samples, and then classify the incremental literature based on the trained samples. Due to the comprehensiveness and application of the railway industry, which includes five major fields: vehicles, machinery, engineering, electricity, and vehicles, the topics of railway science and technology literature generally involve multiple disciplines or interdisciplinary fields, resulting in the classification of railway science and technology literature facing the problem of integration. The content of the literature is usually mixed with information from other fields such as economics, computers, management, engineering, and electricity. For this kind of information, the current manual annotation method is difficult to achieve comprehensive recognition. It relies on cumbersome manual extraction of text features in the railway field, and can only classify relatively simple and obvious railway-related documents, and the effect is unstable, affecting efficiency and user experience. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method and device for batch classification of railway scientific and technological documents, so as to more accurately classify railway scientific and technological documents in batches.

[0004] In a first aspect, the present invention provides a method for batch classification of railway scientific and technological documents, comprising:

[0005] Determine a first railway scientific and technological document feature extraction module of a historical batch and a second railway scientific and technological document feature extraction module of a historical batch, wherein the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module have the same structure and different parameters, and the first railway scientific and technological document feature extraction module of the historical batch and the second railway scientific and technological document feature extraction module of the historical batch are obtained based on document training of the historical batch;

[0006] Inputting the initial documents of the current batch into the first railway scientific and technological document feature extraction module of the historical batch to obtain a first dense feature vector set of each of the initial documents, and inputting the first dense feature vector set into a predetermined out-of-domain scoring module to obtain a first out-of-domain score;

[0007] Inputting the initial documents of the current batch into the second railway scientific and technological document feature extraction module of the historical batch to obtain a second dense feature vector set of each initial document, and inputting the second dense feature vector set into a predetermined out-of-domain scoring module to obtain a second out-of-domain score;

[0008] Merging the first out-of-domain score and the second out-of-domain score of each of the initial documents to obtain a merged out-of-domain score of each of the initial documents;

[0009] Determine, based on the fused out-of-domain scores, the railway scientific and technological documents in the current batch of the initial documents to be classified that are in-domain documents;

[0010] Generate a first sample for each railway scientific and technological document, and use a graph encoder to generate a corresponding second sample for each railway scientific and technological document, wherein the second sample is a positive sample of the first sample corresponding to the same railway scientific and technological document, and the second sample is a negative sample of the first sample corresponding to a different railway scientific and technological document; input the first sample into the feature extraction module of the first railway scientific and technological document of the historical batch to obtain a third dense feature vector set, and input the positive sample and the negative sample of the first sample into the feature extraction module of the second railway scientific and technological document of the historical batch to obtain a fourth dense feature vector set; based on the difference between the third dense feature vector set and the fourth dense feature vector set, determine the difference between the first sample and the positive sample, and the difference between the first sample and the negative sample; optimize the parameters in the feature extraction module of the first railway scientific and technological document of the historical batch by narrowing the difference between the first sample and the positive sample, and widening the difference between the first sample and the negative sample, and optimize the parameters of the feature extraction module of the second railway scientific and technological document of the historical batch based on the historical parameters of the feature extraction module of the first railway scientific and technological document, so as to obtain the feature extraction module of the first railway scientific and technological document of the current batch and the feature extraction module of the second railway scientific and technological document of the current batch;

[0011] Based on the first railway scientific and technological literature feature extraction module and classifier of the current batch, the current batch of railway scientific and technological literature is classified.

[0012] In an optional implementation, the first out-of-domain score is determined based on the following formula: and the second out-of-domain score :

[0013] ;

[0014]

[0015] Among them, the first dense feature vector set is , the second dense feature vector set is , n is the number of initial documents in the current batch, is the mean vector for each class, where , C is the number of classes, is the shared covariance matrix of all classes, for The pseudo-inverse matrix of

[0016] The fusion out-of-domain score is determined based on the following formula:

[0017] ;

[0018] in, is the weight coefficient.

[0019] In an optional implementation manner, generating a first sample for each railway scientific and technological document, and generating a corresponding second sample for each railway scientific and technological document using a graph encoder includes:

[0020] For each of the railway scientific and technological documents, input the railway scientific and technological document and its corresponding classification number hierarchy into a graph encoder to obtain a graph encoding representation of the railway scientific and technological document;

[0021] Performing a scaled dot product attention calculation on the first sample of each railway scientific and technological document and the graph encoding representation, obtaining a partial embedding representation whose correlation with the initial label of the railway scientific and technological document meets the expected level as a second sample, wherein the initial label of the railway scientific and technological document is determined based on the first railway scientific and technological document feature extraction module of the historical batch and the classifier;

[0022] In an optional implementation, determining the railway scientific and technological documents that are in-domain documents in the current batch of the initial documents to be classified based on the fused out-of-domain scores includes:

[0023] The fused out-of-domain score of each of the initial documents is compared with a preset first threshold. If the fused out-of-domain score is greater than the first threshold, the initial document is determined to be an out-of-domain sample, otherwise it is an in-domain sample.

[0024] In an optional implementation, the first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module are pre-trained Chinese BERT models, and the classifier is a linear classifier.

[0025] In an optional implementation, the linear classifier calculates the probability of belonging to each classification number based on the Sigmoid function, and selects the classification number with a probability greater than a second threshold as the prediction result.

[0026] In an optional embodiment, the structure of the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module includes: an input embedding layer and a 12-layer Transformer encoder; the input embedding layer is used to convert words and tokens in the input text into vector representations; the 12-layer Transformer encoder is composed of 12 stacked Transformer encoders, and each Transformer encoder is composed of multiple self-attention mechanism layers and feedforward neural network layers.

[0027] In a second aspect, the present invention provides a batch classification device for railway scientific and technological documents, comprising:

[0028] A determination module is used to determine a first railway scientific and technological document feature extraction module of a historical batch and a second railway scientific and technological document feature extraction module of a historical batch, wherein the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module have the same structure but different parameters, and the first railway scientific and technological document feature extraction module of the historical batch and the second railway scientific and technological document feature extraction module of the historical batch are obtained based on document training of the historical batch;

[0029] A first input module, used for inputting the initial documents of the current batch into the first railway scientific and technological document feature extraction module of the historical batch to obtain a first dense feature vector set of each of the initial documents, and inputting the first dense feature vector set into a predetermined out-of-domain scoring module to obtain a first out-of-domain score;

[0030] A second input module is used to input the initial documents of the current batch into the second railway scientific and technological document feature extraction module of the historical batch to obtain a second dense feature vector set of each initial document, and input the second dense feature vector set into a predetermined out-of-domain scoring module to obtain a second out-of-domain score;

[0031] A fusion module, used for fusing the first out-of-domain score and the second out-of-domain score of each of the initial documents to obtain a fused out-of-domain score of each of the initial documents;

[0032] A judgment module, used for determining the railway scientific and technological documents belonging to the in-domain documents in the current batch of the initial documents to be classified based on the fused out-of-domain scores;

[0033] an optimization module, for generating a first sample for each railway scientific and technological document, and generating a corresponding second sample for each railway scientific and technological document using a graph encoder, wherein the second sample is a positive sample of the first sample corresponding to the same railway scientific and technological document, and the second sample is a negative sample of the first sample corresponding to a different railway scientific and technological document; inputting the first sample into the first railway scientific and technological document feature extraction module of the historical batch to obtain a third dense feature vector set, and inputting the positive sample and the negative sample of the first sample into the second railway scientific and technological document feature extraction module of the historical batch to obtain a fourth dense feature vector set respectively; based on the difference between the third dense feature vector set and the fourth dense feature vector set, determining the difference between the first sample and the positive sample, and the difference between the first sample and the negative sample; optimizing the parameters in the first railway scientific and technological document feature extraction module of the historical batch by narrowing the difference between the first sample and the positive sample, and widening the difference between the first sample and the negative sample, optimizing the parameters of the second railway scientific and technological document feature extraction module of the historical batch based on the historical parameters of the first railway scientific and technological document feature extraction module, and obtaining the first railway scientific and technological document feature extraction module of the current batch and the second railway scientific and technological document feature extraction module of the current batch;

[0034] A classification module is used to classify the current batch of railway scientific and technological documents based on the first railway scientific and technological document feature extraction module and classifier of the current batch.

[0035] In an optional implementation manner, the optimization module is specifically used to:

[0036] For each of the railway scientific and technological documents, input the railway scientific and technological document and its corresponding classification number hierarchy into a graph encoder to obtain a graph encoding representation of the railway scientific and technological document;

[0037] Performing a scaled dot product attention calculation on the first sample of each railway scientific and technological document and the graph encoding representation, obtaining a partial embedding representation whose correlation with the initial label of the railway scientific and technological document meets the expected level as a second sample, wherein the initial label of the railway scientific and technological document is determined based on the first railway scientific and technological document feature extraction module of the historical batch and the classifier;

[0038] In an optional implementation manner, the judgment module is specifically used to:

[0039] The fused out-of-domain score of each of the initial documents is compared with a preset first threshold. If the fused out-of-domain score is greater than the first threshold, the initial document is determined to be an out-of-domain sample, otherwise it is an in-domain sample.

[0040] In an optional implementation, the first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module are pre-trained Chinese BERT models, and the classifier is a linear classifier.

[0041] In an optional implementation, the linear classifier calculates the probability of belonging to each classification number based on the Sigmoid function, and selects the classification number with a probability greater than a second threshold as the prediction result.

[0042] In an optional embodiment, the structure of the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module includes: an input embedding layer and a 12-layer Transformer encoder; the input embedding layer is used to convert words and tokens in the input text into vector representations; the 12-layer Transformer encoder is composed of 12 stacked Transformer encoders, each of which is composed of multiple self-attention mechanism layers and feedforward neural network layers. .

[0043] In a third aspect, the present invention provides a server, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the method described in any one of the aforementioned implementation modes when executing the computer program.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to execute the method described in any one of the aforementioned implementations.

[0045] The embodiments of the present application provide a method and device for batch classification of railway scientific and technological documents. Compared with the traditional scientific and technological document classification system, it will be able to obtain more complete classification information more accurately, improve the effect of railway scientific and technological document classification and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flow chart of a batch classification method for railway scientific and technological documents provided for implementation in this application;

[0048] Figure 2A flow chart of another method for batch classification of railway scientific and technological documents provided for implementation in this application;

[0049] Figure 3 A schematic diagram of the correspondence between the provided labels and classifiers is implemented for this application;

[0050] Figure 4 A flow chart of another method for batch classification of railway scientific and technological documents provided for implementation in this application;

[0051] Figure 5 A schematic diagram of the structure of a batch classification device for railway scientific and technological documents provided for the implementation of this application;

[0052] Figure 6 A flow chart of a method for batch classification of railway scientific and technological documents is provided for the implementation of this application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0054] Figure 1 A schematic flow chart of a batch classification method for railway scientific and technological literature is provided for implementation in this application. Figure 1 As shown, the method includes:

[0055] S110, determining a first railway scientific and technological document feature extraction module of a historical batch and a second railway scientific and technological document feature extraction module of a historical batch, wherein the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module have the same structure but different parameters, and the first railway scientific and technological document feature extraction module of the historical batch and the second railway scientific and technological document feature extraction module of the historical batch are obtained based on document training of the historical batch.

[0056] Among them, the first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module are exactly the same in terms of network structure, and both can include: an input embedding layer and a 12-layer Transformer (a deep learning architecture) encoder.

[0057] The input embedding layer is used to convert words and tokens in the input text into vector representations.

[0058] The 12-layer Transformer encoder is composed of 12 stacked Transformer encoders, each of which consists of multiple self-attention mechanism layers and feed-forward neural network layers.

[0059] The first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module have different parameter updating rules during the training process. Due to the different parameter updating rules, the parameters of the trained modules finally obtained through training optimization are also different.

[0060] Among them, there are multiple options for the parameter update rules of the first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module according to actual needs, and the parameter update rules of the two modules can complement each other. For example, the parameter update of the first railway scientific and technological literature feature extraction module can focus more on learning and optimizing the results, and the parameter update of the second railway scientific and technological literature feature extraction module can focus more on the inheritance of process experience.

[0061] As an example, the parameters of the first railway scientific and technological document feature extraction module are updated based on the first rule; the parameters of the second railway scientific and technological document feature extraction module are updated based on the second rule. The first rule is updated based on a predefined loss function, and the second rule is updated based on the historical parameters of the first railway scientific and technological document feature extraction module. The definitions of the first rule and the second rule are described in detail later and will not be repeated here.

[0062] By training the two modules, it is possible to enhance the module's feature extraction capability and enhance the module's ability to recognize comprehensive and applied railway scientific and technological literature.

[0063] In addition, in the embodiment of the present application, both the first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module need to be retrained based on each batch of railway scientific and technological literature. However, since the input is the initial document, the initial document has a lot of document noise. Directly using it to train the first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module is prone to overfitting or failure to fit. Therefore, the first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module that have been trained in historical batches can be used to filter the initial document.

[0064] S120, input the initial documents of the current batch into the first railway scientific and technological document feature extraction module of the historical batch to obtain a first dense feature vector set for each initial document, and input the first dense feature vector set into a predetermined out-of-domain scoring module to obtain a first out-of-domain score.

[0065] Among them, the initial documents of the current batch are input into the first railway scientific and technological document feature extraction module of the historical batch. The input embedding layer of the module converts the words and tags in the input text into vector representations, and then passes through the 12-layer Transformer encoder of the module to obtain the first dense feature vector set.

[0066] Then, a first out-of-domain score may be calculated based on the first dense feature vector set, wherein the size of the out-of-domain score may be used to indicate the size of the probability that the initial document is an out-of-domain document.

[0067] As an example, the first out-of-domain score can be determined based on the following formula: :

[0068]

[0069] Among them, the first dense feature vector set is , n is the number of initial documents in the current batch, is the mean vector for each class, where , C is the number of classes, is the shared covariance matrix of all classes, for The pseudo-inverse matrix of , C and It can be obtained based on the first railway scientific and technological literature feature extraction module of the historical batch.

[0070] S130, input the initial documents of the current batch into the second railway scientific and technological document feature extraction module of the historical batch to obtain a second dense feature vector set for each initial document, and input the second dense feature vector set into a predetermined out-of-domain scoring module to obtain a second out-of-domain score.

[0071] Among them, the initial documents of the current batch are input into the second railway scientific and technological document feature extraction module of the historical batch. The input embedding layer of the module converts the words and tags in the input text into vector representations, and then passes through the 12-layer Transformer encoder of the module to obtain the second dense feature vector set.

[0072] Then, a first out-of-domain score may be calculated based on the first dense feature vector set.

[0073] As an example, the first out-of-domain score can be determined based on the following formula: :

[0074]

[0075] Among them, the second dense feature vector set is , n is the number of initial documents in the current batch, is the mean vector for each class, where , C is the number of classes, is the shared covariance matrix of all classes, for The pseudo-inverse matrix of , C and It can be obtained based on the second railway scientific and technological literature feature extraction module of the historical batch.

[0076] S140, fusing the first out-of-domain score and the second out-of-domain score of each initial document to obtain a fused out-of-domain score of each initial document.

[0077] In the embodiment of the present application, there are multiple ways to merge the first out-of-domain rating and the second out-of-domain rating, and the fusion can be performed by weighted average. For example, the fused out-of-domain rating can be determined based on the following formula:

[0078]

[0079] in, is the weight coefficient. The size of can be an empirical value, or it can be optimized according to the quality of the classification results. The judgment of the quality of the classification results here can be verified using a validation set with known labels. The size of the fused out-of-domain score is used to indicate the probability that the document belongs to the in-domain document.

[0080] S150, determining the railway scientific and technological documents that belong to the in-domain documents in the current batch of initial documents to be classified based on the fusion out-of-domain scores;

[0081] After determining the above-mentioned fused out-of-domain score, the fused out-of-domain score of each initial document can be compared with a preset first threshold. If the fused out-of-domain score is greater than the first threshold, the initial document is determined to be an out-of-domain sample, otherwise it is an in-domain sample.

[0082] The first threshold value may be an empirical value, which may be determined according to actual needs. It may also be optimized according to the quality of the classification result. The quality of the classification result here may be verified using a verification set with known labels.

[0083] S160, generating a first sample for each railway scientific and technological document, and using a graph encoder to generate a corresponding second sample for each railway scientific and technological document, the second sample is a positive sample of the first sample corresponding to the same railway scientific and technological document, and the second sample is a negative sample of the first sample corresponding to a different railway scientific and technological document; inputting the first sample into a feature extraction module of a first railway scientific and technological document of a historical batch to obtain a third dense feature vector set, and inputting the positive sample and the negative sample of the first sample into a feature extraction module of a second railway scientific and technological document of a historical batch to obtain a fourth dense feature vector set respectively; based on the difference between the third dense feature vector set and the fourth dense feature vector set, determining the difference between the first sample and the positive sample, and the difference between the first sample and the negative sample; optimizing the parameters in the feature extraction module of the first railway scientific and technological document of the historical batch by narrowing the difference between the first sample and the positive sample, and widening the difference between the first sample and the negative sample, optimizing the parameters of the feature extraction module of the second railway scientific and technological document of the historical batch based on the historical parameters of the feature extraction module of the first railway scientific and technological document, and obtaining the feature extraction module of the first railway scientific and technological document of the current batch and the feature extraction module of the second railway scientific and technological document of the current batch.

[0084] Among them, the above-mentioned steps of generating a first sample for each railway scientific and technological document and using a graph encoder to generate a corresponding second sample for each railway scientific and technological document can be achieved through the following steps: for each railway scientific and technological document, the railway scientific and technological document and its corresponding classification number hierarchy are input into the graph encoder to obtain a graph encoding representation of the railway scientific and technological document; the first sample of each railway scientific and technological document is subjected to a scaled dot product attention calculation with the graph encoding representation to obtain a partial embedding representation whose correlation with the initial label of the railway scientific and technological document is consistent with the expected degree as the second sample, wherein the initial label of the railway scientific and technological document is determined based on the feature extraction module and classifier of the first railway scientific and technological document in the historical batch.

[0085] Among them, the graph encoder is a tool used to encode the label hierarchy and output label features. This application adopts Graphormer. Graphormer introduces the Transformer module in the encoding of the graph and shows good performance on multiple graph-related tasks. The uniqueness of Graphormer is that in addition to introducing the Transformer module, three encoding methods are also adopted, which greatly enhances the representation ability of the graph. The first is centrality encoding, which is used to capture the importance of nodes in the graph. By assigning a learnable vector to each node, the vector is added to the node features of the input layer. The second is spatial encoding, which assigns learnable embeddings to node pairs according to the spatial relationship between nodes. The third is edge encoding, which focuses on processing additional spatial information in edge features. This encoding method calculates the dot product mean of the edge features and the learnable embedding on the shortest path, and uses it in the Attention module. Overall, Graphormer models graphs from multiple perspectives and can be easily applied to hierarchical text classification tasks.

[0086] S170, classifying the current batch of railway scientific and technological documents based on the first railway scientific and technological document feature extraction module and classifier of the current batch.

[0087] Among them, the first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module are pre-trained Chinese BERT models, and the classifier is a linear classifier.

[0088] The linear classifier calculates the probability of belonging to each classification number based on the Sigmoid function, and selects the classification number with a probability greater than the second threshold as the prediction result.

[0089] The second threshold value may be an empirical value, which may be determined according to actual needs, or may be obtained through historical data training.

[0090] As an example, Figure 2 As shown, after determining the initial documents to be classified, which usually include some non-railway scientific and technological documents, all the initial documents can be first input into a feature extraction module based on historical batches of railway scientific and technological documents training to perform feature extraction, and the extracted features are input into the out-of-domain scoring module to obtain a fused out-of-domain score.

[0091] Based on the fusion out-of-domain score, the initial documents can be screened to obtain this batch of railway scientific and technological documents.

[0092] After determining the railway scientific and technological documents of this batch, the feature extraction module of the historical batch can be optimized based on the predetermined training rules, and the railway scientific and technological documents of this batch can be classified based on the optimized feature extraction module and classifier to obtain the classification result.

[0093] Specifically, the feature extraction module may include multiple modules, and the multiple modules may have the same structure but different parameters. The multiple modules may be optimized based on different parameter optimization rules.

[0094] For example, Figure 2 As shown, the feature extraction module may include a first railway scientific and technological document feature extraction module and a second railway scientific and technological document feature extraction module, and the parameters of the second railway scientific and technological document feature extraction module are obtained based on the historical parameters of the first railway scientific and technological document feature extraction module. Based on this, the initial document can be input into a railway scientific and technological document feature extraction module and a second railway scientific and technological document feature extraction module respectively to obtain a first dense feature vector set and a second dense feature vector set, and then the first dense feature vector set and the second dense feature vector set are respectively input into the out-of-domain scoring module to obtain a first out-of-domain score and a second out-of-domain score. The first out-of-domain score and the second out-of-domain score are fused to obtain a fused out-of-domain score. Based on this, the obtained fused out-of-domain score can learn the common features between similar instances and distinguish the differences between non-similar instances, making the model and its optimization simpler and more generalized, especially for the railway scientific and technological document input fusion discipline, there are many technical overlaps with other technical fields, and identification by this method can enhance the recognition ability for railway scientific and technological documents and the ability to distinguish the categories of railway scientific and technological documents.

[0095] In some embodiments, the parameters of the first railway scientific and technological document feature extraction module are recorded as Q, and the parameters of the second railway scientific and technological document feature extraction module are recorded as K. Alternatively, the feature extraction module may also include P railway scientific and technological document feature extraction modules; the parameters of the Lth railway scientific and technological document feature extraction module are recorded as Q, and the parameters of the L+1th railway scientific and technological document feature extraction module are recorded as K. Wherein, K=K*w+(1-w)Q.

[0096] Among them, for the optimization of Q, the received text pairs can be used as input, and the classification probability of each label can be output in the final classification head, and compared with the classification label to obtain the classification loss, and the model parameters can be optimized according to the classification loss. Finally, a trained classification model is obtained.

[0097] In some embodiments, the classification system of the railway scientific and technological literature can be pre-established, or it can be automatically classified and summarized based on technical keywords.

[0098] As an example, Figure 3As shown, the classification system may include multi-level labels, wherein the first-level labels may include seven labels of “vehicle, machine, worker, electricity, vehicle, supply, communication, and passenger”, each first-level label may correspond to one or more second-level labels, and each second-level label may also correspond to one or more third-level labels.

[0099] For example, the second-level labels corresponding to the first-level label "machine" may include: "steam engine, internal combustion engine, electric engine, gas turbine, hybrid" and so on; the third-level labels corresponding to the second-level label "steam engine" may include: "general issues, boiler part, steam engine part, brake and auxiliary" and so on; the fourth-level labels corresponding to the third-level label "boiler part" may include: "fire box, boiler body, smoke box" and so on.

[0100] In some embodiments, Figure 3 As shown, the classifier may include multiple sub-classifiers, each sub-classifier corresponds to a level of label. For example, all first-level labels may correspond to a first-level sub-classifier, and the first-level classifier is used to determine the probability that the input data conforms to each first-level label. If the first-level label corresponds to multiple second-level labels, then the second-level label may correspond to a second-level sub-classifier, and the second-level sub-classifier is used to determine the probability that the input data conforms to each second-level label corresponding to the first-level label. If the second-level label corresponds to multiple third-level labels, then the third-level label may correspond to a third-level sub-classifier, and the third-level sub-classifier is used to determine the probability that the input data conforms to each third-level label corresponding to the second-level label.

[0101] For example, the first sub-classifier may correspond to the first label level, and the probability sequence of each label output by the first sub-classifier corresponds to the probability of each first label of the first label level, and the first label level may be a first-level label; under the first sub-classifier, corresponding to each first label, there may also be a corresponding second sub-classifier, and the probability sequence of each label output by the second sub-classifier corresponds to the probability of each second label of the next level corresponding to the first label, and so on, the multiple levels of sub-classifiers may have a tree structure.

[0102] In specific use, the target railway scientific and technological document can be first input into the first railway scientific and technological document feature extraction module of the current batch, and the features extracted by the first railway scientific and technological document feature extraction module of the current batch are input into the highest-level sub-classifier (corresponding to the first-level label), and the first label probability sequence is output. Based on the first label probability sequence, the target first-level label is determined; based on the target first-level label, the second-highest-level sub-classifier corresponding to the target first-level label is determined, and the features extracted by the first railway scientific and technological document feature extraction module of the current batch are input into the second-highest-level sub-classifier, and the second label probability sequence is output. Based on the second label probability sequence, the target second-level label is determined, and so on, until the output result of the lowest-level sub-classifier is obtained, and the third label is determined based on the output result, and the target label sequence corresponding to the target railway scientific and technological document is determined based on the first label, the second label, the third label and any intermediate-level labels.

[0103] In some embodiments, a document generally includes a document title, abstract, main text, drawings, and document bibliographic items, etc., and the document title, abstract, main text, drawings, and document bibliographic items, etc. may contain some keywords associated with the category.

[0104] For example, for the title of a document, it usually includes keywords such as the technical field to which it belongs and the key technologies used. The document can be classified directly based on the title. However, the naming method of a document is usually not fixed, so the usefulness of the key information carried in the name is usually uneven. There is a certain deviation when only using the name for classification. Therefore, a comprehensive evaluation can be conducted by comprehensively considering the contents of the abstract, text, drawings, and document entries, so as to obtain a more accurate classification result.

[0105] In the embodiment of the present application, each document can be divided into a plurality of short sentence combinations, and each short sentence can be corresponding to a first weight, which is determined by the position of the short sentence in the document. For example, the weights of various positions in the document can be predetermined, wherein the weight of the document name is greater than the weight of the abstract, greater than the weight of the text, greater than the weight of the document entry, and greater than the weight of the attached figure.

[0106] Entity recognition can be performed for each short sentence, and the entity is mainly used to refer to keywords related to classification. For example, for the short sentence "Wireless communication method based on railway application scenario", the short sentence can be segmented by word segmentation tools such as Jieba to obtain the word segmentation result "Wireless communication method based on railway application scenario". Based on the word segmentation result, entity recognition can be performed to obtain "railway, application, scenario, wireless communication"

[0107] The short sentence and the entity can be concatenated to obtain "[CLS] short sentence [SEP] entity [SEP]" as the input of the feature extraction module, where [CLS] and [SEP] can be considered as a word, and each Chinese character can also be considered as a word.

[0108] A third-party Chinese BERT pre-trained language model can be used to input the input data of "[CLS] short sentence [SEP] entity [SEP]" into the language model to obtain a context vector representation. For example, the vector of the first word "[CLS]" in the sentence can be used to feed into a linear layer to map the 768-dimensional vector of BERT into a 2-dimensional vector, and then pass through the classifier to obtain the probability distribution of the sequence. Among them, the pre-trained language model can be fine-tuned and the classifier can be trained according to the current batch of railway scientific and technological documents. After determining that the recognition ability of the classification model meets the requirements based on the loss function, the optimized language model and classifier can be considered as the trained classification model. In addition, in the process of training the classification model, the cross entropy can be used to determine the loss function of the model. For example, based on the loss function, the classification model can be trained for multiple cycles, and the optimized classification model with the highest accuracy on the validation set is selected as the final trained classification model. Based on the trained classification model, the target text can be judged to determine whether the target text is an opinion text. In the specific judgment, the target text can be used as the input of the trained classification model, and the output is the judgment result.

[0109] Based on this, the sub-classification result of each short sentence can be obtained, and the classification results of each short sentence are merged based on the first weight of each short sentence to obtain the final classification result of the document.

[0110] For example, Figure 4 As shown, for the initial document, it can be split into short sentences and the first weight of each short sentence is determined, for example, Figure 4 As shown, the document is divided into the first short sentence, the second short sentence, ... the nth short sentence, where n may be the number of all short sentences included in the document.

[0111] Then, each short sentence is input into the railway science and technology literature feature extraction module of the historical batch to obtain the first feature, the second feature...the nth feature (each feature is a set of dense feature vectors); each feature is input into the out-of-domain scoring module to obtain the first fusion score, the second fusion score...the nth fusion score, and the first fusion score, the second fusion score...the nth fusion score are weighted averaged based on the first weight of each short sentence to obtain a secondary fusion score. The railway science and technology literature belonging to the in-domain literature in the current batch of initial documents to be classified can be determined based on the secondary fusion out-of-domain score. Among them, the first feature, the second feature...the nth feature can include two sub-features, respectively, and the two sub-features are the outputs of the first railway science and technology literature feature extraction module of the historical batch and the second railway science and technology literature feature extraction module of the historical batch.

[0112] Railway scientific and technological documents may be divided into short sentences and the first weight of each short sentence may be determined. For example, railway scientific and technological documents may be divided into the first short sentence, the second short sentence, ... the nth short sentence, where n may be the number of all short sentences included in the document.

[0113] Then, each short sentence is input into the first railway scientific and technological literature feature extraction module of the historical batch to obtain the first feature, the second feature...the nth feature (each feature is a set of dense feature vectors); each feature is input into the classifier to obtain the first classification result, the second classification result...the nth classification result, and the first classification result, the second classification result...the nth classification result are weighted averaged based on the first weight of each short sentence to obtain a fused classification result. The railway scientific and technological literature can be determined based on the fused classification result, wherein the classifier can include multiple levels, and each feature can be input into each level of sub-classifier respectively.

[0114] Figure 5 A schematic diagram of the structure of a batch classification device for railway scientific and technological documents provided in an embodiment of the present application. Figure 5 As shown, the device comprises:

[0115] Determining module 501, used to determine a first railway scientific and technological document feature extraction module of a historical batch and a second railway scientific and technological document feature extraction module of a historical batch, wherein the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module have the same structure but different parameters, and the first railway scientific and technological document feature extraction module of the historical batch and the second railway scientific and technological document feature extraction module of the historical batch are obtained based on document training of the historical batch;

[0116] A first input module 502 is used to input the initial documents of the current batch into the first railway scientific and technological document feature extraction module of the historical batch to obtain a first dense feature vector set of each initial document, and input the first dense feature vector set into a predetermined out-of-domain scoring module to obtain a first out-of-domain score;

[0117] The second input module 503 is used to input the initial documents of the current batch into the second railway scientific and technological document feature extraction module of the historical batch to obtain a second dense feature vector set of each initial document, and input the second dense feature vector set into a predetermined out-of-domain scoring module to obtain a second out-of-domain score;

[0118] A fusion module 504 is used to fuse the first out-of-domain score and the second out-of-domain score of each initial document to obtain a fused out-of-domain score of each initial document;

[0119] A judgment module 505 is used to determine the railway scientific and technological documents that belong to the in-domain documents in the current batch of initial documents to be classified based on the fusion out-of-domain scores;

[0120] The optimization module 506 is used to generate a first sample for each railway scientific and technological document, and generate a corresponding second sample for each railway scientific and technological document using a graph encoder, wherein the second sample is a positive sample of the first sample corresponding to the same railway scientific and technological document, and the second sample is a negative sample of the first sample corresponding to a different railway scientific and technological document; the first sample is input into the feature extraction module of the first railway scientific and technological document of the historical batch to obtain a third dense feature vector set, and the positive sample and the negative sample of the first sample are input into the feature extraction module of the second railway scientific and technological document of the historical batch to obtain a fourth dense feature vector set respectively; based on the difference between the third dense feature vector set and the fourth dense feature vector set, the difference between the first sample and the positive sample, and the difference between the first sample and the negative sample are determined; by narrowing the difference between the first sample and the positive sample, and widening the difference between the first sample and the negative sample, the parameters in the feature extraction module of the first railway scientific and technological document of the historical batch are optimized, and the parameters of the feature extraction module of the second railway scientific and technological document of the historical batch are optimized based on the historical parameters of the feature extraction module of the first railway scientific and technological document, so as to obtain the feature extraction module of the first railway scientific and technological document of the current batch and the feature extraction module of the second railway scientific and technological document of the current batch;

[0121] The classification module 507 is used to classify the current batch of railway scientific and technological documents based on the first railway scientific and technological document feature extraction module and the classifier of the current batch.

[0122] In some embodiments, the optimization module 506 is specifically configured to:

[0123] For each railway scientific and technological document, the railway scientific and technological document and its corresponding classification number hierarchy system are input into the graph encoder to obtain the graph encoding representation of the railway scientific and technological document;

[0124] Performing a scaled dot product attention calculation on the first sample of each railway scientific and technological document and the graph encoding representation, obtaining a partial embedding representation whose correlation with the initial label of the railway scientific and technological document meets the expected level as the second sample, wherein the initial label of the railway scientific and technological document is determined based on the feature extraction module and classifier of the first railway scientific and technological document of the historical batch;

[0125] In some embodiments, the determination module 505 is specifically used to:

[0126] The fused out-of-domain score of each initial document is compared with a preset first threshold. If the fused out-of-domain score is greater than the first threshold, the initial document is determined to be an out-of-domain sample, otherwise it is an in-domain sample.

[0127] In some embodiments, the first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module are pre-trained Chinese BERT models, and the classifiers are linear classifiers.

[0128] In some embodiments, the linear classifier calculates the probability of belonging to each classification number based on the Sigmoid function, and selects the classification number with a probability greater than a second threshold as the prediction result.

[0129] In some embodiments, the structures of the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module include: an input embedding layer and a 12-layer Transformer encoder; the input embedding layer is used to convert words and tokens in the input text into vector representations; the 12-layer Transformer encoder is composed of 12 stacked Transformer encoders, and each Transformer encoder is composed of multiple self-attention mechanism layers and feedforward neural network layers.

[0130] See also Figure 6 As shown, the server 600 provided in the embodiment of the present application includes at least: a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program, the batch classification method of railway scientific and technological documents provided in the embodiment of the present application is implemented.

[0131] The server 600 provided in the embodiment of the present application may also include a bus 603 connecting different components (including the processor 601 and the memory 602). The bus 603 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, and the like.

[0132] The memory 602 may include a readable storage medium in the form of a volatile memory, such as a random access memory (RAM) 6021 and / or a cache memory 6022, and may further include a read-only memory (ROM) 6023. The memory 602 may also include a program tool 6025 having a group (at least one) of program modules 6024, and the program modules 6024 include but are not limited to an operating subsystem, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment.

[0133] The processor 601 may be a processing element or a collective term for multiple processing elements. For example, the processor 601 may be a central processing unit (CPU) or one or more integrated circuits configured to implement the batch classification method for railway scientific and technological documents provided in the embodiment of the present application. Specifically, the processor 601 may be a general-purpose processor, including but not limited to a CPU, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0134] The server 600 can communicate with one or more external devices 604 (e.g., keyboards, remote controls, etc.), and can also communicate with one or more devices that enable users to interact with the server 600 (e.g., mobile phones, computers, etc.), and / or communicate with any device that enables the server 600 to communicate with one or more other servers 600 (e.g., routers, modems, etc.). Such communication can be performed through an input / output (I / O) interface 605. In addition, the server 600 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 606. Figure 6 As shown, the network adapter 606 communicates with other modules of the server 600 via the bus 603. It should be understood that although Figure 6Not shown, other hardware and / or software modules may be used in conjunction with server 600, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) subsystems, tape drives, and data backup storage subsystems.

[0135] It should be noted that Figure 6 The server 600 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0136] The computer-readable storage medium provided in the embodiment of the present application is introduced below. The computer-readable storage medium provided in the embodiment of the present application stores computer instructions, and when the computer instructions are executed by the processor, the batch classification method of railway scientific and technological documents provided in the embodiment of the present application is implemented. Specifically, the computer instructions can be built-in or installed in the processor, so that the processor can implement the batch classification method of railway scientific and technological documents provided in the embodiment of the present application by executing the built-in or installed computer instructions.

[0137] In addition, the batch classification method of railway scientific and technological documents provided in the embodiment of the present application can also be implemented as a computer program product, which includes a program code. When the program code is run on a processor, it implements the batch classification method of railway scientific and technological documents provided in the embodiment of the present application.

[0138] The computer program product provided in the embodiments of the present application may adopt one or more computer-readable storage media, and the computer-readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any suitable combination of the above. Specifically, more specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, RAM, ROM, Erasable Programmable Read Only Memory (EPROM), optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0139] The computer program product provided in the embodiment of the present application may adopt a CD-ROM and include program code, and may also be run on a server such as a road management device. However, the computer program product provided in the embodiment of the present application is not limited thereto. In the embodiment of the present application, the computer-readable storage medium may be any tangible medium containing or storing program code, and the program code may be used by or in combination with an instruction execution system, apparatus, or device.

[0140] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0141] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0142] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0143] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A batch classification method for railway scientific and technological documents, characterized in that: include: Determine a first railway scientific and technological document feature extraction module of a historical batch and a second railway scientific and technological document feature extraction module of a historical batch, wherein the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module have the same structure and different parameters, and the first railway scientific and technological document feature extraction module of the historical batch and the second railway scientific and technological document feature extraction module of the historical batch are obtained based on document training of the historical batch; Inputting the initial documents of the current batch into the first railway scientific and technological document feature extraction module of the historical batch to obtain a first dense feature vector set of each of the initial documents, and inputting the first dense feature vector set into a predetermined out-of-domain scoring module to obtain a first out-of-domain score; Inputting the initial documents of the current batch into the second railway scientific and technological document feature extraction module of the historical batch to obtain a second dense feature vector set of each initial document, and inputting the second dense feature vector set into a predetermined out-of-domain scoring module to obtain a second out-of-domain score; Merging the first out-of-domain score and the second out-of-domain score of each of the initial documents to obtain a merged out-of-domain score of each of the initial documents; Determine, based on the fused out-of-domain scores, the railway scientific and technological documents in the current batch of the initial documents to be classified that are in-domain documents; Generate a first sample for each railway scientific and technological document, and use a graph encoder to generate a corresponding second sample for each railway scientific and technological document, wherein the second sample is a positive sample of the first sample corresponding to the same railway scientific and technological document, and the second sample is a negative sample of the first sample corresponding to a different railway scientific and technological document; input the first sample into the feature extraction module of the first railway scientific and technological document of the historical batch to obtain a third dense feature vector set, and input the positive sample and the negative sample of the first sample into the feature extraction module of the second railway scientific and technological document of the historical batch to obtain a fourth dense feature vector set; based on the difference between the third dense feature vector set and the fourth dense feature vector set, determine the difference between the first sample and the positive sample, and the difference between the first sample and the negative sample; optimize the parameters in the feature extraction module of the first railway scientific and technological document of the historical batch by narrowing the difference between the first sample and the positive sample, and widening the difference between the first sample and the negative sample, and optimize the parameters of the feature extraction module of the second railway scientific and technological document of the historical batch based on the historical parameters of the feature extraction module of the first railway scientific and technological document, so as to obtain the feature extraction module of the first railway scientific and technological document of the current batch and the feature extraction module of the second railway scientific and technological document of the current batch; Classifying the current batch of railway scientific and technological documents based on the first railway scientific and technological document feature extraction module and classifier of the current batch; The first out-of-domain score is determined based on the following formula and the second out-of-domain score : ; Among them, the first dense feature vector set is , the second dense feature vector set is , n is the number of initial documents in the current batch, is the mean vector for each class, where , C is the number of classes, is the shared covariance matrix of all classes, for The pseudo-inverse matrix of The fusion out-of-domain score is determined based on the following formula: ; in, is the weight coefficient.

2. The method according to claim 1, characterized in that The step of generating a first sample for each railway scientific and technological document and generating a corresponding second sample for each railway scientific and technological document using a graph encoder includes: For each of the railway scientific and technological documents, input the railway scientific and technological document and its corresponding classification number hierarchy into a graph encoder to obtain a graph encoding representation of the railway scientific and technological document; A scaled dot product attention calculation is performed on the first sample of each railway scientific and technological document and the graph encoding representation to obtain a partial embedding representation that meets the expected correlation with the initial label of the railway scientific and technological document as the second sample, wherein the initial label of the railway scientific and technological document is determined based on the first railway scientific and technological document feature extraction module of the historical batch and the classifier.

3. The method according to claim 1, characterized in that Determining the railway scientific and technological documents belonging to the in-domain documents in the current batch of the initial documents to be classified based on the fused out-of-domain scores includes: The fused out-of-domain score of each of the initial documents is compared with a preset first threshold. If the fused out-of-domain score is greater than the first threshold, the initial document is determined to be an out-of-domain sample, otherwise it is an in-domain sample.

4. The method according to claim 1, characterized in that: The first railway scientific and technological literature feature extraction module and the second railway scientific and technological literature feature extraction module are pre-trained Chinese BERT models, and the classifier is a linear classifier.

5. The method according to claim 4, characterized in that The linear classifier calculates the probability of belonging to each classification number based on the Sigmoid function, and selects the classification number with a probability greater than a second threshold as the prediction result.

6. The method according to claim 4, characterized in that The structures of the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module include: an input embedding layer and a 12-layer Transformer encoder; the input embedding layer is used to convert words and tags in the input text into vector representations; the 12-layer Transformer encoder is composed of 12 stacked Transformer encoders, and each Transformer encoder is composed of multiple self-attention mechanism layers and feedforward neural network layers.

7. A batch classification device for railway scientific and technological documents, characterized in that: include: A determination module is used to determine a first railway scientific and technological document feature extraction module of a historical batch and a second railway scientific and technological document feature extraction module of a historical batch, wherein the first railway scientific and technological document feature extraction module and the second railway scientific and technological document feature extraction module have the same structure but different parameters, and the first railway scientific and technological document feature extraction module of the historical batch and the second railway scientific and technological document feature extraction module of the historical batch are obtained based on document training of the historical batch; A first input module, used for inputting the initial documents of the current batch into the first railway scientific and technological document feature extraction module of the historical batch to obtain a first dense feature vector set of each of the initial documents, and inputting the first dense feature vector set into a predetermined out-of-domain scoring module to obtain a first out-of-domain score; A second input module is used to input the initial documents of the current batch into the second railway scientific and technological document feature extraction module of the historical batch to obtain a second dense feature vector set of each initial document, and input the second dense feature vector set into a predetermined out-of-domain scoring module to obtain a second out-of-domain score; Among them, the first out-of-domain rating and the second out-of-domain score Determined based on the following formula: ; Among them, the first dense feature vector set is , the second dense feature vector set is , n is the number of initial documents in the current batch, is the mean vector for each class, where , C is the number of classes, is the shared covariance matrix of all classes, for The pseudo-inverse matrix of A fusion module is used to fuse the first out-of-domain score and the second out-of-domain score of each of the initial documents to obtain a fused out-of-domain score of each of the initial documents; the fused out-of-domain score is determined based on the following formula: ; in, is the weight coefficient; A judgment module, used for determining the railway scientific and technological documents belonging to the in-domain documents in the current batch of the initial documents to be classified based on the fused out-of-domain scores; an optimization module, for generating a first sample for each railway scientific and technological document, and generating a corresponding second sample for each railway scientific and technological document using a graph encoder, wherein the second sample is a positive sample of the first sample corresponding to the same railway scientific and technological document, and the second sample is a negative sample of the first sample corresponding to a different railway scientific and technological document; inputting the first sample into the first railway scientific and technological document feature extraction module of the historical batch to obtain a third dense feature vector set, and inputting the positive sample and the negative sample of the first sample into the second railway scientific and technological document feature extraction module of the historical batch to obtain a fourth dense feature vector set respectively; based on the difference between the third dense feature vector set and the fourth dense feature vector set, determining the difference between the first sample and the positive sample, and the difference between the first sample and the negative sample; optimizing the parameters in the first railway scientific and technological document feature extraction module of the historical batch by narrowing the difference between the first sample and the positive sample, and widening the difference between the first sample and the negative sample, optimizing the parameters of the second railway scientific and technological document feature extraction module of the historical batch based on the historical parameters of the first railway scientific and technological document feature extraction module, and obtaining the first railway scientific and technological document feature extraction module of the current batch and the second railway scientific and technological document feature extraction module of the current batch; A classification module is used to classify the current batch of railway scientific and technological documents based on the first railway scientific and technological document feature extraction module and classifier of the current batch.

8. A server comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method according to any one of claims 1 to 6.