Meeting minutes classification method, device, computer equipment and storage medium

Abstract

The present application relates to the field of text classification, and in particular to a method, device, computer equipment and storage medium for classifying meeting minutes. The method comprises: extracting sentence-level features of meeting minutes samples to obtain a sentence-level feature set; obtaining sentence-level feature subsets corresponding to each section of the meeting minutes sample in the sentence-level feature set, and generating document representations of corresponding sections based on each sentence-level feature subset; based on each document representation, using the cross entropy of each category weight of the meeting minutes sample as the objective function for training optimization to perform model training to obtain a meeting minutes classification model; using the meeting minutes classification model to classify the meeting minutes to be classified to obtain classification results. The present invention realizes the automation of meeting minutes classification by training the meeting minutes classification model.

Description

Technical Field

[0001] The present application relates to the field of text classification, and in particular to a method, apparatus, computer equipment and storage medium for classifying meeting minutes. Background Art

[0002] Meeting minutes are official documents compiled according to the purpose, agenda, resolutions, etc. of the meeting. They are records of the basic circumstances of the meeting, making it convenient for reporting to superiors or communicating and distributing to relevant personnel.

[0003] In the prior art, the meeting minutes are classified by manually reading the meeting minutes. When the number of meeting minutes is large, the efficiency of manual classification is low and the cost is high. Summary of the invention

[0004] Based on this, it is necessary to provide a meeting minutes classification method, device, computer equipment and storage medium to address the above technical issues.

[0005] In a first aspect, an embodiment of the present invention provides a method for classifying meeting minutes, the method comprising:

[0006] Extract sentence-level features of the meeting minutes sample to obtain a sentence-level feature set;

[0007] Obtaining sentence-level feature subsets corresponding to each section of the meeting minutes sample in the sentence-level feature set, and generating document representations of corresponding sections based on each of the sentence-level feature subsets;

[0008] Based on the document representations, the cross entropy of each category weight of the meeting minutes sample is used as the objective function of training optimization to perform model training to obtain a meeting minutes classification model;

[0009] The meeting minutes classification model is used to classify the meeting minutes to be classified to obtain a classification result.

[0010] In one embodiment, extracting sentence-level features of the meeting minutes sample to obtain a sentence-level feature set includes:

[0011] Extracting word-level semantic features of the meeting minutes sample using a pre-trained model;

[0012] Based on the word-level semantic features, obtaining the sentence-level features;

[0013] Based on the sentence-level features, the sentence-level feature set is obtained.

[0014] In one embodiment, generating document representations of corresponding sections based on each of the sentence-level feature subsets includes:

[0015] Constructing an improved multi-layer Transformer model, the improved multi-layer Transformer model includes a plurality of sequentially connected improved Transformer layers and a first feedforward neural network layer FFNN1, each of the improved Transformer layers includes a sequentially connected SPAttention layer with a multi-head sparse attention mechanism added, a second feedforward neural network layer FFNN2, a linear layer Linear and a normalization layer LayerNorm;

[0016] Each of the sentence-level feature subsets passes through each of the improved multi-layer Transformer layers and the first feed-forward neural network layers in sequence to generate a document representation of the corresponding section.

[0017] In one embodiment, the calculation formula of the document representation S is:

[0018] S = Tanh(Avg(Z)×W t )

[0019] in, represents the weight matrix of the linear transformation, Represents the output of multiple improved Transformer layers, Avg represents average pooling processing, d e Represents the dimension of the sentence-level feature.

[0020] In one embodiment, based on each of the document representations, the cross entropy of each category weight of the meeting minutes sample is used as the objective function of training optimization to perform model training, and the meeting minutes classification model is obtained, including:

[0021] Freeze the weights of the pre-trained model, and use the Softmax function to calculate the probability value of the meeting minutes sample belonging to each category based on the document representation, weight matrix and bias matrix;

[0022] The weighted cross entropy loss is calculated according to the probability values ​​combined with the distribution ratio of each category until it converges.

[0023] In one embodiment, the meeting minutes classification model includes:

[0024]

[0025]

[0026] where y∈[0,1,...,z] is the subscript of the number of meeting minutes categories, t y The unnormalized score of the y-th class, weight[y] represents the weighting of the loss for the y-th class.

[0027] In one embodiment, the method further comprises:

[0028] The optimization result is optimized according to the prior data distribution of the meeting minutes sample.

[0029] In a second aspect, an embodiment of the present invention provides a device for classifying meeting minutes, the device comprising:

[0030] An extraction module is used to extract sentence-level features of the meeting minutes sample and obtain a sentence-level feature set;

[0031] A generation module, used to obtain sentence-level feature subsets corresponding to each section of the meeting minutes sample in the sentence-level feature set, and generate document representations of corresponding sections based on each sentence-level feature subset;

[0032] A training module, for performing model training based on the document representations and taking the cross entropy of each category weight of the meeting minutes sample as the objective function of training optimization to obtain a meeting minutes classification model;

[0033] The classification module is used to classify the meeting minutes to be classified using the meeting minutes classification model to obtain a classification result.

[0034] In a third aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps described in the first aspect.

[0035] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the processor implements the steps described in the first aspect when executing the computer program.

[0036] Compared with the prior art, the above method, apparatus, computer equipment and storage medium extract sentence-level features of the meeting minutes sample to obtain a sentence-level feature set, obtain the sentence-level feature subsets corresponding to each section of the meeting minutes sample in the sentence-level feature set, and generate document representations of the corresponding sections based on each sentence-level feature subset, and use the cross entropy of each category weight of the meeting minutes sample as the objective function of training optimization to perform model training based on each document representation to obtain a meeting minutes classification model, and use the meeting minutes classification model to classify the meeting minutes to be classified to obtain classification results. The present invention realizes the automation of meeting minutes classification by training the meeting minutes classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a schematic diagram of the structure of a terminal in an embodiment;

[0038] Figure 2A schematic diagram of a flow chart of a method for classifying meeting minutes in one embodiment;

[0039] Figure 3 A schematic diagram of a flow chart of a method for acquiring a sentence-level feature set in one embodiment;

[0040] Figure 4 A schematic diagram of the structure of a meeting minutes classification model in one embodiment;

[0041] Figure 5 A schematic diagram of a process flow of a document representation generation method in one embodiment;

[0042] Figure 6 Schematic diagram of the structure of an improved multi-layer Transformer model in one embodiment;

[0043] Figure 7 A schematic diagram of a model training method in an embodiment

[0044] Figure 8 This is a schematic diagram of module connections of a meeting minutes classification device in one embodiment;

[0045] Fig. 9 Schematic diagram of the structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of the present invention. For ordinary technicians in this field, the present invention can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0047] As shown in the present invention and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0048] Although the present invention makes various references to certain modules in the apparatus according to the embodiments of the present invention, any number of different modules can be used and run on a computing device and / or processor. The modules are only illustrative, and different aspects of the apparatus and method can use different modules.

[0049] It should be understood that when a unit or module is described as being "connected," "coupled" to other units, modules, or blocks, it may refer to being directly connected or coupled, or communicating with other units, modules, or blocks, or there may be intermediate units, modules, or blocks, unless the context clearly indicates otherwise. The term "and / or" as used herein may include any and all combinations of one or more of the associated listed items.

[0050] The meeting minutes classification method provided in this application can be applied to Figure 1 In the terminal shown, the terminal obtains a target detection image, which includes a power transmission composite insulator; the target detection image is input into a trained target detection model to obtain a target area image of the power transmission composite insulator; the target area image is input into a trained contrast learning network model to obtain a contamination recognition result of the power transmission composite insulator. Figure 1 As shown, the terminal may include one or two ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 2 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0051] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the meeting minutes classification method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0053] like Figure 2 As shown, the embodiment of the present invention provides a meeting minutes classification method, which is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0054] S202: extracting sentence-level features of the meeting minutes sample to obtain a sentence-level feature set;

[0055] S204: Obtain sentence-level feature subsets corresponding to each section of the meeting minutes sample in the sentence-level feature set, and generate document representations of corresponding sections based on each of the sentence-level feature subsets;

[0056] S206: Based on the document representations, the cross entropy of each category weight of the meeting minutes sample is used as the objective function of training optimization to perform model training to obtain a meeting minutes classification model;

[0057] S208: Classify the meeting minutes to be classified using the meeting minutes classification model to obtain a classification result.

[0058] Based on the above steps S202-S208, sentence-level features of the meeting minutes sample are extracted to obtain a sentence-level feature set, sentence-level feature subsets corresponding to each section of the meeting minutes sample in the sentence-level feature set are obtained, and based on each sentence-level feature subset, document representations of the corresponding sections are generated respectively, and based on each document representation, the cross entropy of each category weight of the meeting minutes sample is used as the objective function of training optimization to perform model training, and a meeting minutes classification model is obtained, and the meeting minutes classification model is used to classify the meeting minutes to be classified to obtain a classification result. The present invention realizes the automation of meeting minutes classification by training the meeting minutes classification model.

[0059] In step S202, if Figure 3-4 As shown, the sentence-level features of the meeting minutes sample are extracted to obtain a sentence-level feature set including:

[0060] S302: Extract word-level semantic features of the meeting minutes sample using a pre-trained model.

[0061] Specifically, the pre-trained model RoBERTa-wwm, which has learned broad common semantics, is used to extract word-level semantic features. The pre-trained model is used to solve the problem of poor model convergence due to insufficient training data.

[0062] S304: Obtain the sentence-level feature based on the word-level semantic feature.

[0063] S306: Based on the sentence-level features, obtain the sentence-level feature set.

[0064] The text of each section of the meeting minutes sample is processed into sentences, [cls] is used as the semantic extraction position, [sep] is used as the sentence marker, and the sentence-level features E of the meeting minutes sample are extracted. m , we get the sentence-level feature set D, where

[0065] D = (E 1 ,E 2 ,...,E m ) Formula (1)

[0066] In step S204, Figure 5 As shown, the generating of document representations of corresponding sections based on each of the sentence-level feature subsets includes:

[0067] S502: Build an improved multi-layer Transformer model.

[0068] like Figure 6 As shown, the improved multi-layer Transformer model includes a plurality of sequentially connected improved Transformer layers and a first feedforward neural network layer FFNN1 (tanh), and each of the improved Transformer layers includes a sequentially connected SPAttention layer with a multi-head sparse attention mechanism, a second feedforward neural network layer FFNN2, a linear layer Linear, and a normalization layer LayerNorm;

[0069] S504: Each of the sentence-level feature subsets is sequentially passed through each of the improved multi-layer Transformer layers and the first feed-forward neural network layers to generate document representations of corresponding sections.

[0070] Extract sentence-level feature subsets D according to different content sections of the meeting minutes sample nThe improved multi-layer Transformer model is fed into the model, where n represents different sections. By extracting the semantics at the sentence level, the model's sentence-level abstract understanding ability can be greatly improved. Furthermore, the sentence structure information is extracted through the improved Transformer layer containing the multi-head sparse attention mechanism and the first feedforward neural network layer FFNN1 to generate the document representation S.

[0071] The multi-head sparse attention mechanism layer SPAttention can greatly improve the model's computational efficiency while losing a small amount of model prediction performance. Furthermore, by automatically weighting the data of different content sections in the meeting minutes, the model can automatically learn the weight of different content sections (such as: meeting background, purpose requirements, main topics, resolution consensus, etc.) on the classification results, allowing the model to automatically discover the classification weights and further improve the model's recognition accuracy. The specific calculation is as follows:

[0072] The multi-head sparse attention mechanism layer SPAttention is calculated as follows:

[0073] SPAttention(D n ) = LayerNorm (D n + MultiHead(D n )) Formula (2)

[0074] MultiHead(D n ) = Concat(head 1 , head 2 , . . . , head h ) × W O Formula (3)

[0075] Among them, h is the number of attention heads of the multi-head sparse attention mechanism layer SPAttention, LayerNorm is the normalization layer, W O is the weight matrix, R represents the real number set, d e It is the dimension of sentence-level features.

[0076] head m is the attention of the mth head, and the sparse attention calculation mechanism of each head is as follows:

[0077]

[0078]

[0079] head i =head 1i +head 2iFormula (6)

[0080] Among them, k represents the distance between sentences, Q i =D×W i Q , represents the query vector in the sparse attention mechanism layer; K i =D×W i K , represents the key vector in the sparse attention mechanism layer; V i =D×W i V , represents the value vector in the sparse attention mechanism layer, are the weight matrices of the i-th head as the parameters to be learned, is an m×m matrix, where the value of the ath row and jth column represents the attention weight of the ath sentence to the jth sentence, V i Contains features from the sentence, head 1i represents the attention of the current sentence with a distance of nk, n∈{1,2,...,m / k}, head 2i Represents the attention that the relative distance between the current sentence and other sentences does not exceed k.

[0081] The output of the multi-head sparse attention mechanism layer SPAttention is calculated through the second feed-forward neural network layer FFNN2 (Relu), the linear layer and the normalization layer, and the result is:

[0082] D'=LayerNorm(SPAttention(D)+Relu(SPAttention(D)×W r )×W s )formula (7)

[0084] in W r , W s are weight matrices as parameters to be learned.

[0085] D' is fed into multiple identical improved Transformer layers to form a matrix Z for calculating the document representation S of each section. The document representation S calculation formula is:

[0086] S = Tanh(Avg(Z)×W t ) Formula (8)

[0087] in, represents the weight matrix of the linear transformation, Represents the output of multiple consecutive improved Transformer layers, Avg represents average pooling processing, d eRepresents the dimension of the sentence-level feature.

[0088] In step S206, if Figure 7 As shown, based on the document representation, the cross entropy of each category weight of the meeting minutes sample is used as the objective function of training optimization to perform model training, and the meeting minutes classification model is obtained, including:

[0089] S702: Freeze the weights of the pre-trained model, and use the Softmax function to calculate the probability value of the meeting minutes sample belonging to each category based on the document representation, weight matrix and bias matrix;

[0090] S704: Calculate a weighted cross entropy loss according to the probability value combined with the distribution ratio of each category until it converges.

[0091] In the training phase, the weights of the pre-trained model RoBERTa-wwm are frozen and the text representation of each section is S n (For example, the conference background indicates S 1 、Purpose requirement expression S 2 、Main topic representationS 3 , resolution consensus representation S 4 ) are stacked into a column and multiplied by the weight matrix w i , add bias matrix b i , the cross entropy of each category weight is increased according to the distribution of meeting minutes sample data as the training optimization objective function for training, and the training meeting minutes classification model is obtained. The calculation method is as follows:

[0092]

[0093]

[0094] where y∈[0,1,...,z] is the subscript of the number of meeting minutes categories, t y The unnormalized score of the y-th class, weight[y] represents the weighting of the loss for the y-th class.

[0095] The specific method of weighted cross entropy loss is to add a penalty coefficient greater than 1 to the minority class cross entropy to increase the cost of misclassifying the class. During the training process, after each round (or after N rounds), the test results are obtained on the validation set, and the best validation set accuracy so far is recorded. As the number of training rounds increases, if the test error of the model on the validation set increases after multiple rounds of training, the training is stopped.

[0096] In step S208, after obtaining the meeting minutes to be classified, data cleaning is performed on them to remove format placeholders including but not limited to "\t", "\r", "\n", "\s" and other invisible characters that serve as placeholders and format control.

[0097] Furthermore, data enhancement is performed on the time and names of non-main participants in the classified meeting minutes. The time and names of non-main participants are randomly replaced to generate more samples for model training.

[0098] In one embodiment, the method further comprises:

[0099] The optimization result is optimized according to the prior data distribution of the meeting minutes sample.

[0100] By leveraging the prior distribution of meeting minutes samples to optimize prediction results, the different data distribution characteristics of meeting minutes categories in different application scenarios are automatically learned, and the accuracy of meeting minutes classification can be improved with only a minimal increase in computational effort.

[0101] The specific steps are as follows:

[0102] S802: Select a threshold α. If the meeting minutes prediction result P is less than α, the result can be regarded as high confidence. If the meeting minutes prediction result P is greater than or equal to α, the result is regarded as low confidence.

[0103] S804: The results with high confidence in the meeting minutes prediction data are set as the first n, and the remaining Nn results are low confidence. For the low confidence results, the high confidence part is used as a reference to correct them. Specifically:

[0104] The low confidence result P (j) With high confidence result P (1,2,3...,n) Standardization, even if all new P (k) The mean vector is equal to the prior distribution P ave ;

[0105] P (k) ←P (k) / P ave ×P last Formula (11)

[0106]

[0107]

[0108] Among them, k∈{1,2,...,n}∪{j}, j∈{n+1,n+2,...,N}, N is the total amount of data to be predicted, n is the amount of data with high confidence, P (k)is the probability distribution of a result in the low confidence result set.

[0109] S806: To ensure the uniformity, P (k) Standardize, that is:

[0110]

[0111] Among them, m is the number of label categories, P i (k) P (k) The probability of a certain label.

[0112] S808: Alternately iterate and execute steps S804-S806 until the results converge, and keep the latest P (j) As the prediction result of the original j-th meeting minutes sample. According to formulas (11) and (14), P (k) Iterate and update until a stable P is obtained (k) .

[0113] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0114] In one embodiment, if Figure 8 As shown, the present invention provides a meeting minutes classification device, the device comprising:

[0115] An extraction module 802 is used to extract sentence-level features of the meeting minutes sample to obtain a sentence-level feature set;

[0116] A generating module 804 is used to obtain sentence-level feature subsets corresponding to each section of the meeting minutes sample in the sentence-level feature set, and generate document representations of corresponding sections based on each sentence-level feature subset;

[0117] A training module 806 is used to perform model training based on each of the document representations and use the cross entropy of each category weight of the meeting minutes sample as a training optimization objective function to obtain a meeting minutes classification model;

[0118] The classification module 808 is used to classify the meeting minutes to be classified using the meeting minutes classification model to obtain a classification result.

[0119] In one embodiment, the extraction module is specifically used for:

[0120] Extracting word-level semantic features of the meeting minutes sample using a pre-trained model;

[0121] Based on the word-level semantic features, obtaining the sentence-level features;

[0122] Based on the sentence-level features, the sentence-level feature set is obtained.

[0123] In one embodiment, the generating module is specifically used for:

[0124] An improved multi-layer Transformer model, wherein the improved multi-layer Transformer model comprises a plurality of sequentially connected improved Transformer layers and a first feedforward neural network layer, wherein each of the improved Transformer layers comprises a sequentially connected SPAttention layer with a multi-head sparse attention mechanism, a second feedforward neural network layer, a linear layer, and a normalization layer;

[0125] Each of the sentence-level feature subsets passes through each of the improved Transformer layers and the first feed-forward neural network layers in sequence to generate a document representation of the corresponding section.

[0126] In one embodiment, the calculation formula of the document representation S is:

[0127] S = Tanh(Avg(Z)×W t )

[0128] in, represents the weight matrix of the linear transformation, represents the output of the improved multi-layer Transformer layer, Avg represents the average pooling process, d e Represents the dimension of the sentence-level feature.

[0129] In one embodiment, the training module is specifically used for:

[0130] Freeze the weights of the pre-trained model, and use the Softmax function to calculate the probability value of the meeting minutes sample belonging to each category based on the document representation, weight matrix and bias matrix;

[0131] The weighted cross entropy loss is calculated according to the probability values ​​combined with the distribution ratio of each category until it converges.

[0132] In one embodiment, the meeting minutes classification model includes:

[0133]

[0134]

[0135] where y∈[0,1,...,z] is the subscript of the number of meeting minutes categories, t y The unnormalized score of the y-th class, weight[y] represents the weighting of the loss for the y-th class.

[0136] In one embodiment, the device further comprises:

[0137] The optimization module is used to optimize the optimization result according to the prior data distribution of the meeting minutes sample.

[0138] For the specific definition of the meeting minutes classification device, please refer to the definition of the meeting minutes classification method above, which will not be repeated here. Each module in the above-mentioned meeting minutes classification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0139] In one embodiment, the present invention provides a computer device, which may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory and a network interface connected by a device bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating device, a computer program and a database. The internal memory provides an environment for the operation of the operating device and the computer program in the non-volatile storage medium. The database of the computer device is used to store motion detection data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in any of the above-mentioned meeting minutes classification method embodiments are implemented.

[0140] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0141] In one embodiment, the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in any of the above-mentioned meeting minutes classification method embodiments are implemented.

[0142] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0143] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A meeting minutes classification method, characterized in that: The method comprises: Extract sentence-level features of the meeting minutes sample to obtain a sentence-level feature set; Obtaining sentence-level feature subsets corresponding to each section of the meeting minutes sample in the sentence-level feature set, and generating document representations of corresponding sections based on each of the sentence-level feature subsets; Based on the document representations, the cross entropy of each category weight of the meeting minutes sample is used as the objective function of training optimization to perform model training to obtain a meeting minutes classification model; Using the meeting minutes classification model to classify the meeting minutes to be classified, and obtaining a classification result; Generating document representations of corresponding sections based on each of the sentence-level feature subsets includes: Constructing an improved multi-layer Transformer model, the improved multi-layer Transformer model includes a plurality of sequentially connected improved Transformer layers and a first feedforward neural network layer FFNN1, each of the improved Transformer layers includes a sequentially connected SPAttention layer with a multi-head sparse attention mechanism added, a second feedforward neural network layer FFNN2, a linear layer Linear and a normalization layer LayerNorm; Each of the sentence-level feature subsets passes through each of the improved multi-layer Transformer layers and the first feed-forward neural network layer in sequence to generate a document representation of the corresponding section; Based on the document representation, the cross entropy of each category weight of the meeting minutes sample is used as the objective function of training optimization to perform model training, and the meeting minutes classification model is obtained, including: Freeze the weights of the pre-trained model, and use the Softmax function to calculate the probability value of the meeting minutes sample belonging to each category based on the document representation, weight matrix and bias matrix; Calculate the weighted cross entropy loss according to the probability value combined with the distribution ratio of each category until it converges; The meeting minutes classification model includes: Among them, y∈[0,1,...,z] is the subscript of the number of meeting minutes categories, t y The unnormalized score of the y-th class, weight[y] represents the weighting of the loss of the y-th class; wi is the weight matrix, bi is the bias matrix, S1...Sn is the text representation of each section; The sentence-level features of the meeting minutes sample are extracted to obtain a sentence-level feature set including: Extracting word-level semantic features of the meeting minutes sample using a pre-trained model; Specifically, we use RoBERTa-wwm, a pre-trained model that has learned a wide range of common semantics, to extract word-level semantic features; Based on the word-level semantic features, obtaining the sentence-level features; Based on the sentence-level features, obtaining the sentence-level feature set; The text of each section of the meeting minutes sample is divided into sentences, [cls] is used as the semantic extraction bit, [sep] is used as the sentence identifier, and the sentence-level features of the meeting minutes sample are extracted to obtain a sentence-level feature set.

2. The method according to claim 1, characterized in that The document indicates that the calculation formula for S is: S=Tanh(Avg(Z)×W t ) in, represents the weight matrix of the linear transformation, represents the output of the improved multi-layer Transformer layer, Avg represents the average pooling process, d e represents the dimension of the sentence-level feature; m is the number of sentence-level features in the sentence-level feature set.

3. The method according to claim 1, characterized in that The method further comprises: The classification result is optimized according to the prior data distribution of the meeting minutes sample.

4. A conference minutes classification device, operating according to a conference minutes classification method according to any one of claims 1 to 3, characterized in that: The device comprises: An extraction module is used to extract sentence-level features of the meeting minutes sample and obtain a sentence-level feature set; A generation module is used to obtain the sentence-level feature subsets corresponding to each section of the meeting minutes sample in the sentence-level feature set, and generate document representations of the corresponding sections based on each of the sentence-level feature subsets; specifically: construct an improved multi-layer Transformer model, the improved multi-layer Transformer model includes a plurality of sequentially connected improved Transformer layers and a first feedforward neural network layer FFNN1, each of the improved Transformer layers includes a sequentially connected SPAttention layer with a multi-head sparse attention mechanism added, a second feedforward neural network layer FFNN2, a linear layer Linear and a normalization layer LayerNorm; each of the sentence-level feature subsets passes through each of the improved multi-layer Transformer layers and the first feedforward neural network layer in turn to generate document representations of the corresponding sections; A training module is used to perform model training based on each of the document representations and use the cross entropy of each category weight of the meeting minutes sample as the objective function of training optimization to obtain a meeting minutes classification model; specifically, freeze the weights of the pre-trained model, and use the Softmax function to calculate the probability value of the meeting minutes sample belonging to each category based on each of the document representations, the weight matrix and the bias matrix; calculate the weighted cross entropy loss according to the probability value combined with the distribution ratio of each category until it converges; The meeting minutes classification model includes: Among them, y∈[0,1,...,z] is the subscript of the number of meeting minutes categories, t y The unnormalized score of the y-th class, weight[y] represents the weighting of the loss of the y-th class; wi is the weight matrix, bi is the bias matrix, S1...Sn is the text representation of each section; The classification module is used to classify the meeting minutes to be classified using the meeting minutes classification model to obtain a classification result.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images