Traffic case abstract extraction method based on enhanced semantic research and judgment
By constructing semantic enhancement character vector set T and using Transformer model and CRF model, the problem of low accuracy in abstract extraction of traffic governance cases is solved, and more accurate abstract generation is achieved.
Patent Information
- Application Number
- CN202411854158.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-03-25
AI Technical Summary
In the prior art, the abstract extraction method of traffic governance cases is not very accurate and cannot effectively deal with a large number of simplified words and abbreviated expressions in cases.
The traffic case abstract extraction method based on enhanced semantic analysis is adopted, and the character vector set T for semantic enhancement is constructed, and the Transformer model and CRF model are used, combined with semantic enhancement and context encoding technology is used to realize keyword extraction and abstract generation of traffic cases.
It improves the accuracy of abstract extraction, can extract keywords more accurately, and generates more accurate abstracts, which are suitable for handling complex language expressions in traffic cases.
Smart Images
Figure CN119691165A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a traffic case summary extraction method based on enhanced semantic analysis. Background Art
[0002] In the process of traffic management cases, it is necessary to carefully extract the key information of the trip summary. However, the large amount of information collected in the case makes it difficult to quickly obtain key information, and manual extraction is obviously too labor-intensive and difficult to implement.
[0003] In recent years, with the development of deep learning, various neural network models have been widely used in automatic text summarization tasks. Extractive summarization technology has made significant progress. The Transformer model has greatly improved the performance of natural language processing tasks due to its attention mechanism and parallel processing capabilities. The Transformer model can not only generate concise summaries through autoregressive generation, but also maintain the information integrity of the original text. It is widely used in the technical field of extractive summarization. However, in actual work, since there is no fixed template for the collected traffic cases, the case writers write them based on the habitual language of their team, which contains a large number of simplified words and abbreviations. Therefore, based on the summary extraction method in the existing technology, the traffic management case is summarized, and the final summary information is not accurate. Summary of the invention
[0004] In order to solve the problem that the summary extraction method in the prior art is not suitable for the summary extraction needs in traffic management cases, the present invention provides a traffic case summary extraction method based on enhanced semantic analysis, which can extract keywords more accurately and generate more precise summaries.
[0005] The technical solution of the present invention is as follows: a traffic case summary extraction method based on enhanced semantic analysis, characterized in that it includes the following steps: S1: Construct a character vector set T for semantic enhancement; For the content to be extracted, construct the semantic enhancement character vector set T; the semantic enhancement character adjacent set T includes the semantic feature classification of the characters in the content to be extracted; The set T contains k subsets, with a total of d character vectors. The set T is represented by C T : ; With C T The central vector set μ corresponding to each subset is expressed as: ; S2: Obtain the article to be extracted, read each sentence, and record it as: sentence to be analyzed; map the characters to be analyzed in the sentence to be analyzed into character vectors; Assume that the article to be extracted corresponds to n characters to be analyzed, then the character vector set s corresponding to the article to be extracted is: ; S3: Each character vector s to be analyzed i Calculate the similarity with the vectors in the center vector set μ and get the vector s i The first two categories with the highest similarity have their corresponding subsets in set T as: set C p and set C q ; C p ={t1,t2,...,t L}; C q ={t1,t2,...,t D}; Where L is the number of character vectors included in the subset corresponding to the category with the highest similarity in the set T, L≤d; D is the number of character vectors included in the subset corresponding to the category with the second highest similarity in the set T, D≤d; S4: Each character vector s to be analyzed i The corresponding categories C p Calculate the similarity of each character vector in the character vector S to be analyzed according to the preset ratio. i n1 characters with high semantic similarity, the character set is denoted as C p '; Each character vector s i The corresponding categories C q Calculate the similarity of each character vector in the character vector s, and obtain the similarity with the character vector s to be analyzed according to the preset ratio. i n2 characters with high semantic similarity, the character set is denoted as C q '; C p ' and C q After the character vectors in ' are merged, they are obtained according to the preset ratio with s i The first m character vector sets with high semantic similarity are denoted as: the most similar set c i : , Among them, Max m Represents the first m values in reverse order from largest to smallest; S5: Construct a summary extraction model based on the Transformer model; Constructing a training data set and a validation data set, and training the summary extraction model using the training data set to obtain the trained summary extraction model; S6: Send the article to be extracted to the trained summary extraction model, and obtain the character vector s to be analyzed from the article to be extracted based on Transformer context encoding i The corresponding weights and hidden states are s i The hidden state is denoted as: h i ; Among them, h i The dimension size is the same as the character vector c ij Same, c ij ∈c i ; j is less than or equal to m; According to the character vector s to be analyzed i The corresponding weight is used to obtain the summary to be judged this time; S7: For each character vector s to be analyzed i Approximate character c ij ∈c i Assign different weights p ij ; ; Among them, p ij Indicates c i The weight of the jth character in ; S8: Put p ij With c ij Perform weighted summation to obtain the enhanced character vector s for each character in the sentence i The semantic information of a i ; Based on the summary to be judged, through a i Represent the summary content and obtain the semantically enhanced summary; The semantically enhanced summary corresponds to a summary encoding set A; A={a1,a2,....,a n'}, where n' is the number of characters included in the summary to be judged; ; S9: Calculate the similarity between the semantically enhanced summary and the article to be extracted to obtain the similarity to be determined; Comparing the similarity to be determined with a preset similarity threshold, and when the threshold to be determined is greater than the similarity threshold, obtaining a final output summary; Otherwise, steps S5 to S9 are executed in a loop.
[0006] Its further features are: The method for producing the summary to be judged and the semantically enhanced summary is: Input the character vector into the CRF model to obtain the final summary sequence; In step S4, the preset ratio is set to 1 / 2; Then, n1=L / 2, n2=D / 2, m=(L+D) / 4; When training the summary extraction model, the optimization objective function used is: ; Among them, x i is the prediction sample, x j is the true label sample, x l is the sample in the data set, l is the total number of samples in the data set, P is the attribute weight, Ω i For x i The set of sample subscripts belonging to the same category; When solving the minimum value of the optimization objective function, the stochastic gradient descent method SGD is used for solving; In step S1, the method for constructing the semantic enhancement character vector set T includes the following steps: a1: The character vector e corresponding to each character in the sentence W obtained by the Bert pre-training model b ; ; Where n represents the length of the sentence, w i It represents the i-th character in the sentence. The input sentence is represented as: W={w1,w2,....,w n}; a2: Convert each character in the sentence into a character vector e by matching the existing character vector set trained by the Word2Vec tool based on the Skip-gram model w , vector representation of characters in the sentence; ; a3: The character vector e obtained by the Bert pre-training model b , and the character vector e obtained by the Word2Vec tool w Perform concatenation, and use the concatenation result X as the final character vector representation to form a set T; ; a4: Cluster the character vectors in set T; Divide d character vectors into k sets based on clustering Then minimize the sum of squares in the set; The objective function is as follows: ; In the formula, C T i Represents a character vector t j The set of characters classified into the i-th category, μ i It is a set C T i The center vector of a5: The result C obtained by clustering the character vector set T T i , C T i The set contains k subsets, denoted as: , With each subset C T i The corresponding center vector μ i The set is denoted as: ; The clustering in step a4 is implemented based on K-means clustering; The similarity calculations in step S3, step S4 and step S9 are all calculated based on cosine similarity.
[0007] The present application provides a traffic case summary extraction method based on enhanced semantic judgment, which constructs the semantic enhancement character vector set T based on historical data, and the set T includes the semantic information of the characters in different scenes. Each character in each sentence in the article to be extracted is judged to belong to the approximate category in the character vector set T based on similarity calculation, that is, after learning the semantic information of the same vocabulary in different scenes, the characters included in the sentence are calculated with cosine similarity with all the character vectors in the category according to the contextual semantics of the sentence, so as to obtain characters with high semantic similarity with the characters in each category, further strengthen the semantic information of each character in the sentence, greatly improve the accuracy of the meaning judgment of each character in the specific scene during the summary extraction process, and thus improve the accuracy of summary extraction. Finally, by setting a similarity threshold, the similarity between the semantically enhanced summary and the article to be extracted is compared with the similarity threshold, and the accuracy of the final output summary and the extracted article is controlled to ensure that keywords can be accurately extracted and a more accurate summary can be generated. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 This is a schematic diagram of the structure of the traffic management case summary extraction system; Figure 2 This is a flow chart of the traffic case summary extraction method based on enhanced semantic analysis in this application; Figure 3 Schematic diagram of the semantic enhancement process. DETAILED DESCRIPTION
[0009] like Figure 1 and Figure 2 As shown, in order to implement the traffic management case summary extraction method in this application, a summary extraction system is first constructed. The summary extraction system includes: a coding layer, a semantic enhancement module, and a research and analysis module. Based on the summary extraction system, a traffic case summary extraction method based on enhanced semantic analysis in this application is implemented, which specifically includes the following steps.
[0010] First, in the encoding layer, a semantic enhancement character vector set T is constructed. The method for constructing the semantic enhancement character vector set T includes the following steps.
[0011] a1: The character vector e corresponding to each character in the sentence W obtained by the Bert pre-training model b ; ; Where n represents the length of the sentence, w i It represents the i-th character in the sentence. The input sentence is represented as: W={w1,w2,....,w n}.
[0012] a2: Convert each character in the sentence into a character vector e by matching the existing character vector set trained by the Word2Vec tool based on the Skip-gram model w , vector representation of characters in the sentence; .
[0013] a3: The character vector e obtained by the Bert pre-training model b , and the character vector e obtained by the Word2Vec tool w Perform concatenation, and use the concatenation result X as the final character vector representation to form a set T; .
[0014] a4: Cluster the character vectors in set T; Divide d character vectors into k sets based on clustering Then minimize the sum of squares in the set; The objective function is as follows: ; In the formula, C T i Represents a character vector t jThe set of characters classified into the i-th category, μ i It is a set C T i The center vector of .
[0015] The clustering in step a4 can be implemented based on any clustering algorithm in the prior art, and in this embodiment, it is implemented based on K-means clustering. Based on the clustering algorithm, all the most similar characters with the same feature are divided into a category group, and subsequent calculations only need to be calculated with the center vector μ, ensuring that the amount of subsequent calculations can be reduced.
[0016] a5: The result C obtained by clustering the character vector set T T i , C T i The set contains k subsets, denoted as: , With each subset C T i The corresponding center vector μ i The set is denoted as: .
[0017] S1: The semantic enhancement module reads the character vector set T for semantic enhancement built based on the encoding layer; Obtaining a semantic enhancement character vector set T constructed in the coding layer; the semantic enhancement character adjacent set T includes semantic feature classification of characters in the content to be extracted; The set T contains k subsets, with a total of d character vectors. The set T is represented by C T : ; With C T The central vector set μ corresponding to each subset is expressed as: .
[0018] S2: The semantic enhancement module reads the article to be extracted, obtains each sentence, which is recorded as the sentence to be analyzed; and maps the characters to be analyzed in the sentence to be analyzed into character vectors; Assume that the article to be extracted corresponds to n characters to be analyzed, then the character vector set s corresponding to the article to be extracted is: .
[0019] S3: Based on the character vector set T, the character vector s to be analyzed i Classification, preliminarily determine the meaning of the input sentence. The specific method is: each character vector to be analyzed s iCalculate the similarity with the vectors in the center vector set μ and get the vector s i The first two categories with the highest similarity have their corresponding subsets in set T as: set C p and set C q ; C p ={t1,t2,...,t L}; C q ={t1,t2,...,t D}; Where L is the number of character vectors included in the subset corresponding to the category with the highest similarity in the set T, L≤d; D is the number of character vectors included in the subset corresponding to the category with the second highest similarity in the set T, D≤d.
[0020] S4: Based on the context, further learn the contextual meaning and strengthen the semantic information of each character. The specific method is: each character vector s i The corresponding categories C p Calculate the similarity of each character vector in the character vector S to be analyzed according to the preset ratio. i n1 characters with high semantic similarity, the character set is denoted as C p '; Each character vector s i The corresponding categories C q Calculate the similarity of each character vector in the character vector s, and obtain the similarity with the character vector s to be analyzed according to the preset ratio. i n2 characters with high semantic similarity, the character set is denoted as C q '; C p ' and C q After merging the character vectors in ', we get the top m character vector sets with the highest semantic similarity to s according to the preset ratio, which is recorded as: the most similar set c i : , Among them, Max m Represents the first m values in descending order.
[0021] The specific preset ratio in step S4 is preset according to the accuracy of case extraction. In this embodiment, the preset ratio is set to 1 / 2; Then, n1=L / 2, n2=D / 2, m=(L+D) / 4; ; .
[0022] The semantic enhancement operation performed in the semantic enhancement module is shown in the accompanying drawings of the specification. Figure 3 After reading the character vector set T, according to step S3, for each character vector s to be analyzed i , perform similarity calculation, and find the similar category to which they belong in set T: set C p and set C q However, in order to strengthen the semantic information of each character in the sentence, further learning is needed. i The contextual semantics of the sentence to which the carrier gas belongs, so in step S4, the character s i Compute cosine similarity with all character vectors in the same category.
[0023] In this method, the semantic enhancement operation performed in the semantic enhancement module is used to learn the rich contextual features of the characters in the input article. i In Category C p and C q Find the character sets with the highest phase velocity respectively, and record them as: C p ' and C q ', C p ' and C q 'After mixing together, sort them from large to small according to similarity, and then extract the characters with the highest similarity to form the most similar set c i , ensuring that the similar characters extracted by this method are similar to s i The meaning is closest to that of the original text, thereby ensuring that the true meaning of simplified words and abbreviations in the traffic cases in the context can be learned through the semantic enhancement process, making the final summary closer to the original meaning.
[0024] S5: Construct a summary extraction model based on the Transformer model; Construct a training data set and a validation data set, train the summary extraction model through the training data set, and obtain a trained summary extraction model.
[0025] S6: Send the article to be extracted into the trained summary extraction model, and obtain the character vector s to be analyzed from the article to be extracted based on Transformer context encoding i The corresponding weights and hidden states are s i The hidden state is denoted as: h i ; Among them, h i The dimension size is the same as the character vector c ij Same, c ij ∈c i ; j is less than or equal to m; According to the character vector s to be analyzed i The corresponding weight is used to obtain the summary to be judged this time.
[0026] S7: For each character vector s to be analyzed i Approximate character c ij ∈c i Assign different weights p ij ; ; Among them, p ij Indicates c i The weight of the j-th character in .
[0027] S8: Put p ij With c ij Perform weighted summation to obtain the enhanced character vector s for each character in the sentence i The semantic information of a i ; Based on the summary to be judged, through a i Represent the summary content and obtain the semantically enhanced summary; The semantically enhanced summary corresponds to the summary encoding set A; A={a1,a2,....,a n'}, where n' is the number of characters in the abstract to be judged, and n' is less than or equal to the total number of characters n in the article to be extracted; .
[0028] In the semantic enhancement module of this application, an attention mechanism is introduced. For each character vector s to be analyzed, i , for each approximate character c ij Different weights are assigned to weight similar semantic characters according to their contribution in different contexts, ensuring that the semantic information of each character can be strengthened based on contextual semantics and context, and the meanings of various abbreviations and simplified words in traffic cases can be effectively learned.
[0029] S9: Based on the research and analysis module, the semantically enhanced summary and the article to be extracted are similarly calculated to obtain the similarity to be judged; The similarity to be judged is compared with the preset similarity threshold. When the threshold to be judged is greater than the similarity threshold, the final output summary is obtained; Otherwise, steps S5 to S9 are executed in a loop.
[0030] In this method, the summary content is constructed based on the CRF model and the Transformer model output. That is, the production method of the summary to be judged and the summary after semantic enhancement is: Input the character vector into the CRF model to obtain the final summary sequence.
[0031] For example: Input the summary code set A into CRF to obtain the final summary sequence abstract=CRF(A).
[0032] CRFConditional Random Field (CRF) is a discriminative probability model in the prior art, which is used to predict the corresponding output label sequence for a given input sequence. It captures the contextual information of sequence data by considering the dependencies between labels.
[0033] In this method, the minimum cross entropy is used to train the CRF network. The learning rate of the language model is set to 4*10 -4 , the number of hidden layers is set to 512, and the mini batch used is 16.
[0034] In this method, in the judgment and analysis module, the extracted summary and the original text are analyzed again for similarity to determine whether the generated semantic module is similar to the meaning expressed in the original text, so that the final output summary content is closer to the true description of the input text. The specific similarity threshold is set according to the actual accuracy required, and is set to 0.78 in this embodiment.
[0035] For summary texts that do not meet the threshold requirements, steps S5 to S9 are executed in a loop, and context compression extraction is performed again through the summary extraction model until the threshold accuracy requirements are met.
[0036] When training the summary extraction model, the optimization objective function used is: ; Among them, x i is the prediction sample, x j is the true label sample, x l is the sample in the data set, l is the total number of samples in the data set, P is the attribute weight, Ω i For x i The set of sample subscripts belonging to the same category; when solving the minimum value of the optimization objective function, the stochastic gradient descent method SGD is used to solve it.
[0037] Regarding the data set, suppose there are l samples x in the data set l The dataset is divided into two groups: training set and validation set. The samples in the training set are samples with true labels x j .
[0038] In this method, the similarity calculations in step S3, step S4 and step S9 are performed based on a similarity algorithm in the prior art, and in this embodiment are performed based on cosine similarity.
[0039] After using the technical solution of the present invention, a large number of summary generation tasks can be completed without manual description. In order to learn semantic information in more detail, a semantic enhancement module is set in this method to learn the semantic information of the same vocabulary in different scenarios. The research and analysis module is introduced to learn the differences between the real text, so that the generated summary is more accurate. Using the above advantages, the generation of the summary is realized.
Claims
1. A traffic case summary extraction method based on enhanced semantic analysis, characterized in that: It includes the following steps: S1: Construct a character vector set T for semantic enhancement; For the content to be extracted, construct the semantic enhancement character vector set T; the semantic enhancement character adjacent set T includes the semantic feature classification of the characters in the content to be extracted; The set T contains k subsets, with a total of d character vectors. The set T is represented by C T : ； With C T The central vector set μ corresponding to each subset is expressed as: ； S2: Get the article to be extracted, read each sentence, and record it as the sentence to be analyzed; Mapping the characters to be analyzed in the sentence to be analyzed into character vectors; Assume that the article to be extracted corresponds to n characters to be analyzed, then the character vector set s corresponding to the article to be extracted is: ； S3: Each character vector s to be analyzed i Calculate the similarity with the vectors in the center vector set μ and get the vector s i The first two categories with the highest similarity have their corresponding subsets in set T as: set C p and set C q ; C p ={t1,t2,...,t L }; C q ={t1,t2,...,t D }; Where L is the number of character vectors included in the subset corresponding to the category with the highest similarity in the set T, L≤d; D is the number of character vectors included in the subset corresponding to the category with the second highest similarity in the set T, D≤d; S4: Each character vector s to be analyzed i The corresponding categories C p Calculate the similarity of each character vector in the character vector S to be analyzed according to the preset ratio. i n1 characters with high semantic similarity, the character set is denoted as C p '; Each character vector s i The corresponding categories C q Calculate the similarity of each character vector in the character vector s, and obtain the similarity with the character vector s to be analyzed according to the preset ratio. i n2 characters with high semantic similarity, the character set is denoted as C q '; C p ' and C q After the character vectors in ' are merged, they are obtained according to the preset ratio with s i The first m character vector sets with high semantic similarity are denoted as: the most similar set c i : ， Among them, Max m Represents the first m values in reverse order from largest to smallest; S5: Construct a summary extraction model based on the Transformer model; Constructing a training data set and a validation data set, and training the summary extraction model using the training data set to obtain the trained summary extraction model; S6: Send the article to be extracted to the trained summary extraction model, and obtain the character vector s to be analyzed from the article to be extracted based on Transformer context encoding i The corresponding weights and hidden states are s i The hidden state is denoted as: h i ; Among them, h i The dimension size of the character vector c ij Same, c ij ∈c i ; j is less than or equal to m; According to the character vector s to be analyzed i The corresponding weight is used to obtain the summary to be judged this time; S7: For each character vector s to be analyzed i Approximate character c ij ∈c i Assign different weights p ij ; ； Among them, p ij Indicates c i The weight of the jth character in ; S8: Put p ij With c ij Perform weighted summation to obtain the enhanced character vector s for each character in the sentence i The semantic information of a i ; Based on the summary to be judged, through a i Represent the summary content and obtain the semantically enhanced summary; The semantically enhanced summary corresponds to a summary encoding set A; A={a1,a2,....,a n' }, where n' is the number of characters included in the summary to be judged; ； S9: Calculate the similarity between the semantically enhanced summary and the article to be extracted to obtain the similarity to be determined; Comparing the similarity to be determined with a preset similarity threshold, and when the threshold to be determined is greater than the similarity threshold, obtaining a final output summary; Otherwise, steps S5 to S9 are executed in a loop.
2. According to claim 1, a traffic case summary extraction method based on enhanced semantic analysis is characterized by: The method for producing the summary to be judged and the semantically enhanced summary is: Input the character vector into the CRF model to obtain the final summary sequence.
3. According to claim 1, a traffic case summary extraction method based on enhanced semantic analysis is characterized by: In step S4, the preset ratio is set to 1 / 2; Then, n1=L / 2, n2=D / 2, m=(L+D) / 4.
4. According to claim 1, a traffic case summary extraction method based on enhanced semantic analysis is characterized by: When training the summary extraction model, the optimization objective function used is: ； Among them, x i is the prediction sample, x j is the true label sample, x l is the sample in the data set, l is the total number of samples in the data set, P is the attribute weight, Ω i For x i The set of sample subscripts belonging to the same category.
5. The method for extracting traffic case summaries based on enhanced semantic analysis according to claim 1 is characterized by: When solving the minimum value of the optimization objective function, the stochastic gradient descent method SGD is used to solve it.
6. The method for extracting traffic case summaries based on enhanced semantic analysis according to claim 1 is characterized by: In step S1, the method for constructing the semantic enhancement character vector set T includes the following steps: a1: The character vector e corresponding to each character in the sentence W obtained by the Bert pre-training model b ; ； Where n represents the length of the sentence, w i It represents the i-th character in the sentence. The input sentence is represented as: W={w1,w2,....,w n }; a2: Convert each character in the sentence into a character vector e by matching the existing character vector set trained by the Word2Vec tool based on the Skip-gram model w , vector representation of characters in the sentence; ； a3: The character vector e obtained by the Bert pre-training model b , and the character vector e obtained by the Word2Vec tool w Perform concatenation, and use the concatenation result X as the final character vector representation to form a set T; ； a4: Cluster the character vectors in set T; Divide d character vectors into k sets based on clustering Then minimize the sum of squares in the set; The objective function is as follows: ； In the formula, C T i Represents a character vector t j The set of characters classified into the i-th category, μ i It is a set C T i The center vector of a5: The result C obtained by clustering the character vector set T T i , C T i The set contains k subsets, denoted as: ， With each subset C T i The corresponding center vector μ i The set is denoted as: .
7. The method for extracting traffic case summaries based on enhanced semantic analysis according to claim 8 is characterized by: The clustering in step a4 is implemented based on K-means clustering.
8. The method for extracting traffic case summaries based on enhanced semantic analysis according to claim 1 is characterized by: The similarity calculations in step S3, step S4 and step S9 are all calculated based on cosine similarity.