Hybrid time information extraction method and system based on context awareness

Abstract

The invention belongs to the technical field of information extraction, and relates to a hybrid time information extraction method and system based on context awareness. The method comprises the steps of collecting text data containing time information, carrying out context modeling on the time information, and carrying out hybrid time analysis and time expression standardization processing. According to the method, the time information in the text is identified and analyzed more accurately by combining rule matching with the deep learning model, and the accuracy of time information extraction and the adaptability to texts in different fields are improved; through context modeling, the relationship between time expression and context is fully extracted by using an attention mechanism, so that the accuracy of time information analysis is greatly improved; and time information in different forms is unified into a standard format, so that the universality and operability of data are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of information extraction, and in particular, relates to a hybrid time information extraction method and system based on context perception. Background Art

[0002] Time information is ubiquitous in all kinds of text data. In clinical medical record analysis, accurately grasping the time when patients' symptoms appeared and the time of treatment is crucial for diagnosis and treatment decisions; in news event tracking, clarifying the time sequence of events and development can help understand the whole picture of the events; when analyzing legal texts, accurately identifying the terms such as the effective time of the contract and the execution time of the judgment is the basis for ensuring the accurate handling of legal affairs.

[0003] Traditional methods for temporal information extraction mainly include rule-based methods and machine learning-based methods. Rule-based methods rely on predefined rules to identify temporal information, but the real text context is complex and changeable, and there are many implicit temporal expressions. It is difficult for predefined rules to fully cover them, resulting in a significant reduction in the accuracy of extraction in complex contexts. Machine learning methods usually only focus on local features of the text, ignore contextual information, and are sensitive to data distribution. When applied to texts in different fields from the training data, the generalization ability is insufficient and temporal information cannot be accurately extracted. Therefore, it is imperative to develop an algorithm that can overcome the above defects, combine the advantages of rule matching and deep learning models, and improve the accuracy and adaptability of temporal information extraction. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a hybrid time information extraction method and system based on context awareness.

[0005] In a first aspect, the present invention provides a hybrid time information extraction method based on context awareness, including collecting text data containing time information, context modeling of time information, hybrid time parsing and time expression standardization processing; Contextual modeling of temporal information, including training Model and training time relationship reasoning layer; train The model includes: the input text data is processed by the BERT model to obtain the word vector of time information, and the time feature embedding parameters are adjusted at the same time; the model parameters of the multi-head attention mechanism, feedforward network, residual connection and layer normalization are trained; the multi-head attention mechanism is used to extract the relationship between time expression and context; the feedforward network is used to perform feature transformation on the time expression processed by the multi-head attention mechanism; the time expression is used to represent the text data representing time in the input text; The training time relation reasoning layer includes: performing time expression classification using a classifier, predicting the time offset using LSTM, and updating the parameters of the classifier and LSTM; Hybrid time parsing includes: for the parsing of explicit time expressions, matching and adjusting the text time according to regular expression rules; for the parsing of implicit time expressions, training a hybrid model and updating the parameters of the hybrid model; the hybrid model includes a model, a model, and a model.

[0006] In a second aspect, the present invention provides a context-aware hybrid time information extraction system, including a collection unit, a modeling unit, a parsing unit, and a normalization processing unit; The collection unit is used to collect text data containing time information; the modeling unit is used for context modeling of time information; the parsing unit is used for hybrid time parsing, and the normalization processing unit is used for time expression normalization processing; The context modeling of time information includes training a model and training the time relation reasoning layer; Training the model includes: the input text data is processed by the BERT model to obtain the word vectors of time information, and at the same time, the time feature embedding parameters are adjusted; training the model parameters of the multi-head attention mechanism, the feed-forward network, the residual connection, and the layer normalization processing; the multi-head attention mechanism is used to extract the relationship between the time expression and the context; the feed-forward network is used to perform feature transformation on the time expression processed by the multi-head attention mechanism; the time expression is used to represent the text data representing time in the input text; The training time relation reasoning layer includes: performing time expression classification using a classifier, predicting the time offset using LSTM, and updating the parameters of the classifier and LSTM; Hybrid time parsing includes: for the parsing of explicit time expressions, matching and adjusting the text time according to regular expression rules; for the parsing of implicit time expressions, training a hybrid model and updating the parameters of the hybrid model; the hybrid model includes a model, a model, and a model.

[0007] Based on the above technical solutions, the present invention can also be improved as follows.

[0008] Further, the time expression includes absolute time, relative time, and fuzzy time.

[0009] Further, using the multi-head attention mechanism to extract the relationship between the time expression and the context includes: Let be the query vector, is the key vector, is the value vector, is the attention function, is the dimension of the key vector, is the activation function, the query vector is the vector of the current time expression, and the key vector and value vector are the vectors of the context words, then: .

[0010] Furthermore, it is input into the feed-forward network for feature transformation, including: Let and be the weight matrices, is the input of the neural network, and are the bias vectors, is the feed-forward network, is the rectified linear unit activation function, is the maximum function, performs a linear transformation on the input , performs a linear transformation on the result after being processed by the activation function again, and add an offset to the linear transformation result, then: .

[0011] Furthermore, during the model training process, residual connection and layer normalization processing are performed, including: Let be the self-attention function, be the feed-forward network, is the input of the th layer in the neural network, is the time expression after residual connection, is the time expression after normalization processing, is the normalization function, then: ; .

[0012] Furthermore, let the input text be , the category label of the time expression classification be , is the semantic dependency vector of the processed time expression, is the activation function, is the weight matrix, is the bias vector, represents that the category label of the time expression classification is under the condition of the given input text The probability is then: 。

[0013] Furthermore, a hybrid model is adopted to analyze implicit temporal expressions. The input text is passed through model for context feature extraction, model for sequence modeling, and model for boundary recognition to obtain the analysis result of temporal expressions.

[0014] Furthermore, the standardization process of temporal expressions includes relative time analysis and fuzzy time analysis; relative time analysis includes: extracting time units, calculating time offsets, and calculating the target date in combination with the current date; fuzzy time analysis includes: mapping time periods, calculating time ranges, and unifying all temporal expressions into a standard format.

[0015] Furthermore, let the input text be , denote the function of the model, and ; Let the semantic dependency of the temporal expression be , denote the function of the model, then: Let be the activation function, denote that the class label of the temporal expression classification is under the condition of the given input text with the probability of denote the function of the model, and be the time, then:

[0016] The beneficial effects of the present invention are: (1) By combining rule matching and deep learning models, the present invention can more accurately identify and analyze temporal information in text, improving the accuracy of temporal information extraction and the adaptability to texts in different fields; (2) Through context modeling, the attention mechanism is used to fully extract the relationship between temporal expressions and the context. Compared with traditional machine learning methods, the context information is utilized more fully, greatly improving the accuracy of temporal information analysis, especially in complex texts and cross-domain texts; (3) By standardizing time expressions, different forms of time information are unified into a standard format, facilitating subsequent data processing, storage, and analysis, providing a standardized data foundation for various applications, and improving the generality and operability of the data. Description of the Drawings

[0017] Figure 1 It is a schematic diagram of the context-aware hybrid time information extraction method provided in Embodiment 1 of the present invention; Figure 2 It is a schematic diagram of the context-aware hybrid time information extraction system provided in Embodiment 2 of the present invention. Detailed Embodiments

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations.

[0019] Embodiment 1 As an embodiment, as shown in the appendix Figure 1 To solve the above technical problems, this embodiment provides a context-aware hybrid time information extraction method, including collecting text data containing time information, context modeling of time information, hybrid time parsing, and time expression standardization processing; The context modeling of time information includes training the model and the training time relationship inference layer; Training The model includes: the input text data is processed by a BERT (Bidirectional Encoder Representations from Transformers) model to obtain word vectors of time information, and at the same time, the time feature embedding parameters are adjusted; the model parameters of the multi-head attention mechanism, feed-forward network, residual connection, and layer normalization processing are trained; the multi-head attention mechanism is used to extract the relationship between time expressions and the context; the feed-forward network is used to perform feature transformation on the time expressions processed by the multi-head attention mechanism; the time expression is used to represent the text data representing time in the input text; Training the time relationship inference layer includes: using a classifier for time expression classification, using an LSTM (Long Short-Term Memory) to predict the time offset, and updating the parameters of the classifier and the LSTM; Hybrid time parsing, including: for the parsing of explicit time expressions, matching and adjusting the text time according to regular expression rules; for the parsing of implicit time expressions, training a hybrid model and updating the parameters of the hybrid model; the hybrid model includes model, model and model.

[0020] The present invention combines rule matching and a deep learning model to more accurately identify and parse time information in text, improving the accuracy of time information extraction and the adaptability to texts in different fields.

[0021] The present invention models the context of time information and uses the attention mechanism to extract the context relationship of time expressions, thereby improving the accuracy of time information parsing.

[0022] The present invention combines the advantages of rule matching and a deep learning model. Rule matching can quickly and accurately identify explicit time expressions with a fixed format, while the deep learning model effectively processes implicit time expressions in complex contexts through context modeling and semantic understanding, improving the comprehensiveness and accuracy of time information extraction.

[0023] Optionally, time expressions include absolute time, relative time, and fuzzy time.

[0024] Optionally, using the multi-head attention mechanism to extract the relationship between time expressions and context, including: Let be the query vector, be the key vector, be the value vector, be the attention function, be the dimension of the key vector, be the activation function. The query vector is the vector of the current time expression, and the key vector and value vector are the vectors of context words, then: .

[0025] Optionally, inputting to a feed-forward network for feature transformation, including: Let and be the weight matrices, be the input of the neural network, and be the bias vectors, be the feed-forward network, be the rectified linear unit activation function, be the maximum function, Performing a linear transformation on the input , Performing The result after the activation function processing is linearly transformed again. and An offset is added to the linear transformation result, then: .

[0026] Optionally, during the model training process, residual connection and layer normalization processing are performed, including: Let be the self-attention function, be the feed-forward network, be the input of the th layer in the neural network, be the time expression after residual connection, be the time expression after normalization processing, be the normalization function, then: ; .

[0027] Adopting the method of residual connection and layer normalization can improve the stability of model training.

[0028] Optionally, let the input text be , the category label of the time expression classification be , be the processed time expression semantic dependency vector, be the activation function, be the weight matrix, be the bias vector, represents that given the input text , the probability that the category label of the time expression classification is is: .

[0029] In time expression classification, absolute time refers to a clear and fixed time, such as "March 4, 2025", relative time refers to the time compared with the current time, such as "three days later", and there is text unrelated to time information.

[0030] The LSTM is used to predict the time offset. For example: The current date is March 4, 2025, and "three days later" is parsed as March 7, 2025.

[0031] Optionally, a hybrid model is used to parse implicit time expressions. The input text is passed through the model for context feature extraction, the model for sequence modeling, and the model for boundary recognition to obtain the parsing result of the time expression.

[0032] Explicit time expressions usually appear in the text in a fixed format and are matched using regular expressions. For holidays such as "Spring Festival" and "New Year's Day", a keyword matching method is used to match explicit time expressions.

[0033] Optionally, the standardization of time expressions includes relative time parsing and fuzzy time parsing; relative time parsing includes: extracting time units, calculating time offsets, and calculating the target date in combination with the current date; fuzzy time parsing includes: mapping time periods, calculating time ranges, and unifying all time expressions into a standard format.

[0034] The unified standard format is like the format of ISO 8601. The date is expressed as: year - month - day. The year is represented by four digits, and both the month and the date are represented by two digits. If the number is less than two digits, a 0 is added in front. The time is expressed as: "hour:minute:second", such as 14:30:00. The hour, minute, and second are also represented by two digits. The combination of date and time: they are connected by the capital letter T in the middle, for example: 2025 - 03 - 27T14:30:00. Time offsets such as +3 days and -7 days, etc. +3 days means pushing forward 3 days based on the original time, and "-7 days" means going back 7 days based on the original time. The mapped time period such as "evening" is mapped to the time period: 18:00:00 - 23:59:59). The process of standardizing time expressions unifies different forms of time information into the ISO 8601 standard format, which is convenient for subsequent data processing, storage, and analysis, provides a standardized data foundation for various applications, and improves the universality and operability of data.

[0035] Optionally, let the input text be , represent the function of the model, and ; Let the semantic dependency of the time expression be , represent the function of the model, then: ; Let be the activation function, represent that the class label of the time expression classification is under the condition of the given input text with a probability of represent the function of the model, and .

[0036] The present invention is applied in the medical field. By extracting the course time information of patients from electronic medical records, it can assist in clinical decision-making. Applied in the news analysis field, it constructs a news event timeline and can analyze the evolution process of events. Applied in the legal text parsing field, it can identify time clauses in contracts and judgments, improving the efficiency of legal retrieval. Applied in the intelligent customer service and Q&A field, it can enhance the intelligent customer service's understanding and answering ability for time-related questions.

[0037] Embodiment 2 Based on the same principle as the method shown in Embodiment 1 of the present invention, as shown in the appendix Figure 2 In the embodiments of the present invention, a context-aware hybrid time information extraction system is further provided, including a collection unit, a modeling unit, a parsing unit, and a standardization processing unit; The collection unit is used to collect text data containing time information; the modeling unit is used for context modeling of time information; the parsing unit is used for hybrid time parsing, and the standardization processing unit is used for standardization processing of time expressions; The context modeling of time information includes training models and training time relationship inference layers; Training models include: the input text data is processed by the BERT model to obtain word vectors of time information, and at the same time, the time feature embedding parameters are adjusted; the model parameters of the multi-head attention mechanism, feed-forward network, residual connection, and layer normalization processing are trained; the multi-head attention mechanism is used to extract the relationship between time expressions and the context; the feed-forward network is used for feature transformation of the time expressions processed by the multi-head attention mechanism; time expressions are used to represent the text data representing time in the input text; Training the time relationship inference layer includes: using a classifier for time expression classification, using LSTM to predict time offsets, and updating the parameters of the classifier and LSTM; Hybrid time parsing includes: for the parsing of explicit time expressions, matching and adjusting the text time according to regular expression rules; for the parsing of implicit time expressions, training a hybrid model and updating the parameters of the hybrid model; the hybrid model includes models, models, and models.

[0038] Optionally, time expressions include absolute time, relative time, and fuzzy time.

[0039] Optionally, using the multi-head attention mechanism to extract the relationship between time expressions and the context includes: Let be the query vector, be the key vector, be the value vector, is the attention function, is the dimension of the key vector, is the activation function, the query vector is the vector of the current time expression, and the key vector and the value vector are the vectors of the context words, then: .

[0040] Optionally, input to the feed-forward network for feature transformation, including: Let and be the weight matrices, is the input of the neural network, and are the bias vectors, is the feed-forward network, is the rectified linear unit activation function, is the maximum function, Performs a linear transformation on the input , Performs another linear transformation on the result after passing through the activation function, and Add an offset to the linear transformation result, then: .

[0041] Optionally, during the model training process, perform residual connection and layer normalization processing, including: Let be the self-attention function, be the feed-forward network, is the input of the th layer in the neural network, is the time expression after residual connection, is the time expression after normalization processing, is the normalization function, then: ; .

[0042] Optionally, let the input text be , the class label of the time expression classification is , is the processed semantic dependency vector of the time expression, is the activation function, is the weight matrix, is the bias vector, represents the probability that the class label of the time expression classification is given the input text, then: .

[0043] Optionally, a hybrid model is used to parse implicit time expressions. The input text is passed through the model for context feature extraction, and the model for sequence modeling, and the model for boundary recognition to obtain the parsing result of the time expression.

[0044] Optionally, the time expression normalization process includes relative time parsing and fuzzy time parsing; relative time parsing includes: extracting time units, calculating time offsets, and calculating target dates in combination with the current date; fuzzy time parsing includes: mapping time periods, calculating time ranges, and unifying all time expressions into a standard format.

[0045] Optionally, let the input text be , representing the function of the model, and being the context vector of the time expression. The extraction of the context vector of the time expression is represented as: ; Let the semantic dependency of the time expression be , representing the function of the model, then: ; Let be the activation function, representing that the class label of the time expression classification is under the condition of the given input text with a probability of , and representing the function of the model. If is the time, then:

[0046] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A hybrid temporal information extraction method based on context awareness, characterized in that: It includes the collection of text data containing time information, context modeling of time information, hybrid time parsing and time expression standardization; Contextual modeling of temporal information, including training Model and training time relationship reasoning layer; train The model includes: the input text data is processed by the BERT model to obtain the word vector of time information, and the time feature embedding parameters are adjusted at the same time; the model parameters of the multi-head attention mechanism, feedforward network, residual connection and layer normalization are trained; the multi-head attention mechanism is used to extract the relationship between time expression and context; the feedforward network is used to perform feature transformation on the time expression processed by the multi-head attention mechanism; the time expression is used to represent the text data representing time in the input text; Training the temporal relationship reasoning layer, including: using a classifier to classify temporal expressions, using LSTM to predict temporal offsets, and updating the parameters of the classifier and LSTM; Hybrid time parsing, including: parsing explicit time expressions, matching and adjusting text time according to regular expression rules; parsing implicit time expressions, training hybrid models, and updating hybrid model parameters; hybrid models include Model, Model and Model.

2. The context-aware hybrid temporal information extraction method according to claim 1, characterized in that: Time expression includes absolute time, relative time and fuzzy time.

3. The context-aware hybrid temporal information extraction method according to claim 1, characterized in that: The multi-head attention mechanism is used to extract the relationship between temporal expression and context, including: set up is the query vector, is the key vector, is a value vector, is the attention function, is the dimension of the key vector, is the activation function, the query vector is the vector expressed at the current time, the key vector and the value vector are the vectors of the context words, then: 。 4. The context-aware hybrid temporal information extraction method according to claim 1, characterized in that: Input to the feedforward network for feature transformation, including: and is the weight matrix, is the input of the neural network, and is the bias vector, is a feed-forward network, is the rectified linear unit activation function, is the maximum value function, For input Perform a linear transformation, About The result after activation function processing is linearly transformed again. and Add an offset to the linear transformation result: 。 5. The context-aware hybrid temporal information extraction method according to claim 1, characterized in that: During the model training process, residual connection and layer normalization are performed, including: is the self-attention function, is a feed-forward network, The first The input of the layer, is the time expression after residual connection, is the normalized time expression, is a normalized function, then: ； 。 6. The context-aware hybrid time information extraction method according to claim 1 is characterized in that: Enter text as , the category label of time expression classification is , is the processed temporal expression semantic dependency vector, is the activation function, is the weight matrix, is the bias vector, Indicates that given the input text In the case of time expression classification, the category label is The probability of , then: 。 7. The context-aware hybrid temporal information extraction method according to claim 1, characterized in that: A hybrid model is used to parse implicit time expressions, and the input text is passed through The model extracts contextual features using The model performs sequence modeling using The model performs boundary recognition and obtains the analytical result of time expression.

8. The context-aware hybrid temporal information extraction method according to claim 1, characterized in that: Time expression standardization includes relative time resolution and fuzzy time resolution; Relative time parsing includes: extracting time units, calculating time offsets, and calculating target dates in combination with the current date; fuzzy time parsing includes: mapping time periods, calculating time ranges, and unifying all time expressions into a standard format.

9. The context-aware hybrid time information extraction method according to claim 1 is characterized in that: Enter text as , express The function of the model, is the context vector of time expression, and the context vector of time expression is extracted as follows: ； Assume that the semantic dependency of time expression is , express The function of the model is: ; set up is the activation function, Indicates that given the input text In the case of time expression classification, the category label is The probability of express The function of the model, is time, then: .

10. A hybrid temporal information extraction system based on context awareness, characterized in that: It includes a collection unit, a modeling unit, an analysis unit and a standardization processing unit; The collection unit is used to collect text data containing time information; the modeling unit is used for context modeling of time information; the parsing unit is used for hybrid time parsing, and the standardization processing unit is used for time expression standardization processing; Contextual modeling of temporal information, including training Model and training time relationship reasoning layer; train The model includes: the input text data is processed by the BERT model to obtain the word vector of time information, and the time feature embedding parameters are adjusted at the same time; the model parameters of the multi-head attention mechanism, feedforward network, residual connection and layer normalization are trained; the multi-head attention mechanism is used to extract the relationship between time expression and context; the feedforward network is used to perform feature transformation on the time expression processed by the multi-head attention mechanism; the time expression is used to represent the text data representing time in the input text; Training the temporal relationship reasoning layer, including: using a classifier to classify temporal expressions, using LSTM to predict temporal offsets, and updating the parameters of the classifier and LSTM; Hybrid time parsing, including: parsing explicit time expressions, matching and adjusting text time according to regular expression rules; parsing implicit time expressions, training hybrid models, and updating hybrid model parameters; hybrid models include Model, Model and Model.

Images