Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding
Through the method based on event knowledge embedding, the Hanyue cross-language event and time-series data set is constructed and word embedding is carried out, which solves the translation errors and insufficient event understanding of low-resource language Vietnamese in cross-language query expansion, and achieves more efficient Cross-language information retrieval performance.
Patent Information
- Application Number
- CN202411344563.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-09-25
AI Technical Summary
Traditional cross-language query expansion methods have problems such as low-resource languages such as Vietnamese, and the lack of model understanding of events, resulting in poor retrieval performance, especially in Chinese-Vietnamese cross-language query expansion.
Using an event knowledge embedding method, the Chinese cross-language event and time-sequence data set is constructed, and the event subject and side description are transformed into vector space using event and time-sequence word embedding, and cross-language alignment is achieved through bilingual shared semantic dictionary to optimize high-level The alignment quality of semantic space, thus generating cross-language event descriptions.
It significantly improves the performance of cross-language information retrieval and the relevance and accuracy of query expansion tasks, and improves the representation ability of the model in low-resource languages and the attention to event elements. The experimental results show that this method is used to cross-language query expansion tasks in Chinese and Vietnam. The performance is better than the baseline model.
Smart Images

Figure CN118861212B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding, and belongs to the technical field of information retrieval. Background Art
[0002] Chinese-Vietnamese cross-language query expansion is to enrich and expand the query in the target language (such as Vietnamese) at the term level based on the input source language (such as Chinese) query according to the event elements in the query, so as to ensure that the query and the expanded content are consistent in describing the event, and improve the recall rate of cross-language event retrieval. Unlike the general term-level query expansion task, the content of the cross-language query expansion based on event knowledge embedding is based on the event in the query, and further requires that the expanded content and the query content have related event elements. Compared with term-level query expansion, cross-language query expansion enhances the model's discovery of event elements in the query, improves the relevance and accuracy of the expanded content and the event, and further improves the performance of the model in downstream retrieval tasks. It helps users overcome language barriers and accurately retrieve events, so as to be applied to actual scenarios such as communication between different cultures and tracking of emergencies, and promote communication and understanding between different cultures.
[0003] Chinese-Vietnamese cross-language event query expansion is a special cross-language query expansion task. Traditional cross-language query expansion usually has two methods: on the one hand, using machine translation to translate the query content into the target language, and then using the target language to expand the query. However, its effect is limited by the performance of machine translation. For languages with sufficient resources such as Chinese and English, the translation error is small and the effect is better; while for low-resource languages such as Vietnamese, the translation performance is limited, and important entities such as names of people, places, and names of organizations may have translation errors, resulting in unsatisfactory performance of the final expanded retrieval. On the other hand, after the query is expanded in the source language, multilingual word embedding or multilingual pre-trained models are used to map different languages to the same semantic space to achieve cross-language query expansion. However, there are currently some multilingual pre-trained models, such as mBert (Multilingual Bert), which have an imbalanced data distribution between different languages. This imbalance causes these models to show strong representation capabilities and achieve good alignment when dealing with languages with abundant resources, such as Chinese and English. However, the representation effect for low-resource languages such as Chinese and Vietnamese is still unsatisfactory, and directly using multilingual pre-trained models to achieve cross-language query expansion between Chinese and Vietnamese still faces huge challenges. At the same time, traditional query expansion methods have a weak understanding of events. Query expansion through pseudo-relevance feedback only considers the semantic similarity between the expanded content and the query, while cross-language query expansion considers the expanded content as an accurate summary or description of the query event elements. Therefore, the cross-language query expansion task needs to improve on the existing multilingual word embedding model, and enhance the representation ability of low-resource languages and the focus on event elements on the existing basis.
[0004] Therefore, the present invention proposes a Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding. By integrating event knowledge embedding technology, the present invention accurately identifies the optimal correlation distance between the event subject and the side description terms in the vector space of a specific time period, and constructs a bilingual shared semantic dictionary based on this, thereby significantly optimizing the alignment quality of the high-level semantic space. The present invention method is compared with a series of baseline methods, and the experimental results prove the effectiveness of the present invention method in the query expansion task. Summary of the invention
[0005] The technical problem solved by the present invention is: the present invention provides a Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding. The present invention generates a cross-language event description based on the event content of a single event subject through a bilingual shared semantic dictionary, thereby realizing Chinese-Vietnamese cross-language query expansion and improving the performance of cross-language information retrieval.
[0006] The technical solution of the present invention is: a Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding, the method comprising:
[0007] Step 1: Construct a Chinese-Vietnamese cross-language event and time series dataset;
[0008] Step 2: The time series content corresponding to the event is converted into a vector space through the event and time series word embedding part, and the event knowledge is embedded in it. The embedded event knowledge adopts multilateral measurement method and is embedded by describing multiple aspects of the main event;
[0009] Step 3: Based on the embedding of events and time-order words, a Chinese-Vietnamese cross-language public dictionary is constructed to expand event knowledge from monolingual to cross-language. By calculating the similarity matrix between Chinese events and event side descriptions, bilingual alignment is achieved in terms of dimensions.
[0010] Step 4: By calculating the similarity between the query and the expanded content, the query expansion set of the final event subject is obtained.
[0011] Furthermore, in the Step 1, a Chinese-Vietnamese cross-language event and time series dataset is constructed. The dataset is sourced from the Chinese and Vietnamese parts of Wikipedia and Vietnam News Network. Questions in the dataset are used to perform Chinese-Vietnamese cross-language queries, and a scoring system is constructed to screen and obtain the Chinese-Vietnamese cross-language event and time series dataset.
[0012] Furthermore, the Step 1 specifically includes:
[0013] Step 1.1, automatically obtain from Vietnam News Network and Wikipedia through web crawler technology; first collect the URLs of the chronological year pages of Vietnam News Network and Wikipedia;
[0014] Step 1.2, send a fake web page request through the code, and include the user agent and key information in the request package to successfully establish a secure response with the server of the target page;
[0015] Step 1.3. After obtaining the source code of the web page, use the Etree package in Python to parse the specific web page content; next, use the XPath method to collect the anchor coordinates of all required content to collect the specified text in the web page.
[0016] Furthermore, the Step 2 specifically includes the following:
[0017] Step 2.1, based on the Chinese event subject X and its corresponding Vietnamese event side description term set E, as well as the Vietnamese event temporal content T, obtain multiple description terms in the set E as anchor points in the space;
[0018] Step 2.2, use the cosine distance D to calculate the distance between each anchor point in the Chinese event subject X and the Vietnamese event side description term set E:
[0019]
[0020] Step 2.3. Finally, find the optimal position of the Chinese event subject X in the time series vector space, and use the L-BFGS optimization algorithm to minimize the cosine distance D from the anchor point to the event subject X and the mean square error MSE of the distance v between the anchor point and the event subject X after embedding in the time series vector space.
[0021] Furthermore, the Step 3 specifically includes the following:
[0022] Based on the original embedding of the Chinese event subject X and the Vietnamese event side description and temporal content (E+T) axes, the corresponding similarity matrices M X and M (E+T) The two axes of correspond to the same words; the linear transformation matrices of the Chinese event subject X and the Vietnamese event side description and temporal content (E+T) are W X and W (E+T) ;
[0023] Step 3.1. A dictionary is constructed between the Chinese and Vietnamese languages and encoded into a sparse matrix. In this matrix, if the jth word in the target language is the translation of the ith word in the source language, the value of the element Dij in the sparse matrix is 1, otherwise it is 0.
[0024] Step 3.2, for M X and M (E+T) Sort the values in each row to get the sorted result sorted(M X ) and sorted(M (E+T) ); Given a word and its sorted(M X ) in the rows, sorted(M (E+T) ) rows are searched using nearest neighbor search to find their corresponding translations;
[0025] Step 3.3, use the argmax() function to find the best orthogonal mapping that maximizes the similarity of the current dictionary Dic;
[0026] Step 3.4, calculate the mapping embedding XW X W (E+T) The best dictionary on the similarity matrix of (E+T);
[0027] Step 3.5, obtain the event subject expansion event set Ce according to the best dictionary obtained through the best orthogonal mapping of the sparse matrix mentioned above.
[0028] Furthermore, the Step 4 specifically includes the following:
[0029] Step 4.1, according to the event knowledge embedding, by calculating the similarity between the query q and the expanded content, the set event Eq of related events is obtained. For the detected related events , create an extended event set Ce;
[0030] The extended event set Ce contains two types of extended content: (1) extended content that is highly similar to the query q; (2) extended content that is highly similar to the related event e;
[0031] Step 4.2: Score each expansion event c in the expansion event candidate set Ce using a combination of two different methods:
[0032] The first method: Calculate the similarity between the extended event c and the query q and the related event e to obtain the similarity between the extended event c and the related event e. , the similarity between the extended event c and the query q , the similarity between related events e and query q , and according to different weight coefficients, obtain the query of the event subject and the expanded event description score ;
[0033] ;
[0034] The second method is to add the temporal correlation evaluation index TempRel on the basis of the temporal embedding space. This evaluation method will give a higher score to the related event e of the extended event c occurring within time t. The score is:
[0035] ;
[0036] k represents the total number of relevant events e;
[0037] The final scoring method for each expansion event c is:
[0038] ;
[0039] in, , , and is the parameter weight. After obtaining the score of each expansion event c, the expansion events c with high scores are screened out according to the score of each expansion event c and the set threshold, thereby obtaining the final query expansion set.
[0040] The beneficial effects of the present invention are as follows: the present invention first constructs a Chinese-Vietnamese cross-language event and time series data set and trains a Chinese-Vietnamese cross-language event query expansion model; then, a time series vector space is established by embedding events and time series words, and the event subject and the similarity matrix of its description language and time series relationship are matched; finally, the event subject expansion event set is obtained by the best orthogonal mapping of the sparse matrix. The present invention uses different baseline models to conduct experiments on the constructed data set, and the results show that the query capability of the present invention is significantly improved compared with other models. The experimental results verify the effectiveness of the method of the present invention in cross-language information retrieval tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 It is the Chinese-Vietnamese cross-language query expansion model framework based on event knowledge embedding in the present invention;
[0042] Figure 2 The method of the present invention is transferred and applied to each baseline model, and the Rouge-L value comparison bar chart before and after improvement is shown;
[0043] Figure 3 The method of the present invention is migrated and applied to each baseline model, and the NDCG@10 value comparison bar chart before and after improvement is shown;
[0044] Figure 4 The method of the present invention is transferred and applied to each baseline model, and the MAP value comparison bar chart before and after improvement is shown;
[0045] Figure 5 The comparison bar graph of NDCG@10 values and MAP values before and after the ablation experiment was performed on each part using the method of the present invention. DETAILED DESCRIPTION
[0046] Example 1: Figure 1-Figure 5 As shown, a Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding includes:
[0047] Step 1: Construct a Chinese-Vietnamese cross-language event and time series dataset;
[0048] Furthermore, in the Step 1, a Chinese-Vietnamese cross-language event and time series dataset is constructed. The dataset is sourced from the Chinese and Vietnamese parts of Wikipedia and Vietnam News Network. Questions in the dataset are used to perform Chinese-Vietnamese cross-language queries, and a scoring system is constructed to screen and obtain the Chinese-Vietnamese cross-language event and time series dataset.
[0049] Furthermore, the Step 1 specifically includes:
[0050] Step 1.1, automatically obtain from Vietnam News Network and Wikipedia through web crawler technology; first collect the URLs of the chronological year pages of Vietnam News Network and Wikipedia;
[0051] Step 1.2, send a fake web page request through the code, and include the user agent and key information in the request package to successfully establish a secure response with the server of the target page;
[0052] Step 1.3. After obtaining the source code of the web page, use the Etree package in Python to parse the specific web page content; next, use the XPath method to collect the anchor coordinates of all required content to collect the specified text in the web page.
[0053] Step 2: The time series content corresponding to the event is converted into a vector space through the event and time series word embedding part, and the event knowledge is embedded in it. The embedded event knowledge adopts multilateral measurement method and is embedded by describing multiple aspects of the main event;
[0054] As a further solution of the present invention, in Step 2, the event knowledge of multiple side descriptions is embedded in the main event using multilateral measurement, and a temporal vector space of embedded event knowledge is constructed. In this step, it is necessary to first analyze the data type and data content of the input Chinese-Vietnamese cross-language event query expansion model, and then use the distance relationship between the event side description terms and the event subject to locate the event description terms that are highly related to the event subject, and embed them in the temporal space to generate the final expanded content.
[0055] Furthermore, the specific steps of Step 2 are as follows:
[0056] Step 2.1, based on the Chinese event subject X and its corresponding Vietnamese event side description term set E, as well as the Vietnamese event temporal content T, obtain multiple description terms in the set E as anchor points in the space;
[0057] For input analysis: Event subject X, this part of the data focuses on the main summary of the event. It covers the core participants, main actions and basic circumstances of the event, providing the model with the overall background of the event; Vietnamese event side description term set E: In addition to the main information, events are often accompanied by a variety of side descriptions. These descriptions may involve the motivations, impacts, feelings of the participants, etc. of the event. In the model of the present invention, special attention is paid to these side descriptions because they provide the model with richer contextual information and help to understand the full picture of the event more accurately; Time content T: This is a unique element in the model of the present invention. Considering that the event occurs in a specific time context, the time period in which the event occurs is taken into consideration as a key dimension, which includes the specific date, time period and other time-related details of the event.
[0058] Step 2.2, use the cosine distance D to calculate the distance between each anchor point in the Chinese event subject X and the Vietnamese event side description term set E:
[0059]
[0060] Among them, the set of anchor points Satisfies the following formula:
[0061]
[0062] Among them, n is a single anchor point, M T is the vectorized time series vector space;
[0063] Step 2.3. Finally, find the optimal position of the Chinese event subject X in the time series vector space, and use the L-BFGS optimization algorithm to minimize the cosine distance D from the anchor point to the event subject X and the mean square error MSE of the distance v between the anchor point and the event subject X after embedding in the time series vector space.
[0064]
[0065] Step 3: Based on the embedding of events and time-order words, a Chinese-Vietnamese cross-language public dictionary is constructed to expand event knowledge from monolingual to cross-language. By calculating the similarity matrix between Chinese events and event side descriptions, bilingual alignment is achieved in terms of dimensions.
[0066] Furthermore, the Step 3 specifically includes the following:
[0067] Based on the original embedding of the Chinese event subject X and the Vietnamese event side description and temporal content (E+T) axes, the corresponding similarity matrices M X and M (E+T) The two axes of correspond to the same words; the linear transformation matrices of the Chinese event subject X and the Vietnamese event side description and temporal content (E+T) are W X and W (E+T) ;
[0068] Step 3.1. A dictionary is constructed between the Chinese and Vietnamese languages and encoded into a sparse matrix. In this matrix, if the jth word in the target language is the translation of the ith word in the source language, the value of the element Dij in the sparse matrix is 1, otherwise it is 0.
[0069] Step 3.2, for M X and M (E+T) Sort the values in each row to get the sorted result sorted(M X ) and sorted(M (E+T)); Given a word and its sorted(M X ) in the rows, sorted(M (E+T) ) rows are searched using nearest neighbor search to find their corresponding translations;
[0070] When using the nearest neighbor search from the source language to the target language, the search result of the target language = argmax (X * W X )·((E+T) * W (E+T) );
[0071] Step 3.3, use the argmax() function to find the best orthogonal mapping that maximizes the similarity of the current dictionary Dic; Step 3.3 specifically includes:
[0072] Through the two steps Step3.1 and Step3.2, convergence is achieved and the best orthogonal mapping that maximizes the similarity of the current dictionary D is obtained. The results are as follows:
[0073] ;
[0074] W X = U and W (E+T) = V gives the optimal solution, where UV T =X T D(E+T) is X T Singular value decomposition of D(E+T);
[0075] Step 3.4, calculate the mapping embedding XW X W (E+T) The best dictionary on the similarity matrix of (E+T);
[0076] Step 3.5, obtain the event subject expansion event set Ce according to the best dictionary obtained through the best orthogonal mapping of the sparse matrix mentioned above.
[0077] Step 4: By calculating the similarity between the query and the expanded content, the query expansion set of the final event subject is obtained.
[0078] Furthermore, the Step 4 specifically includes the following:
[0079] Step 4.1, according to the event knowledge embedding, by calculating the similarity between the query q and the expanded content, the set event Eq of related events is obtained. For the detected related events , create an extended event set Ce;
[0080] The extended event set Ce contains two types of extended content: (1) extended content that is highly similar to the query q; (2) extended content that is highly similar to the related event e;
[0081] Step 4.2: Score each expansion event c in the expansion event candidate set Ce using a combination of two different methods:
[0082] The first method: Calculate the similarity between the extended event c and the query q and the related event e to obtain the similarity between the extended event c and the related event e. , the similarity between the extended event c and the query q , the similarity between related events e and query q , and according to different weight coefficients, obtain the query of the event subject and the expanded event description score ;
[0083] ;
[0084] The second method is to add the temporal correlation evaluation index TempRel on the basis of the temporal embedding space. This evaluation method will give a higher score to the related event e of the extended event c occurring within time t. The score is:
[0085] ;
[0086] k represents the total number of relevant events e;
[0087] The final scoring method for each expansion event c is:
[0088] ;
[0089] in, , , and is the parameter weight. After obtaining the score of each expansion event c, the expansion events c with high scores are screened out according to the score of each expansion event c and the set threshold, thereby obtaining the final query expansion set.
[0090] The above method proposed in the present invention is experimented on a Chinese-Vietnamese cross-language event and time series dataset. The training dataset of the present invention uses the constructed event subject-event side description-time series content dataset for word embedding training. The data volume is 13786, and the training set, test set and validation set are divided into training set, test set and validation set in a ratio of 7:2:1.
[0091] The experimental environment of the present invention is built on the deep learning framework Pytorch1.7.1 based on Python3.7 under Windows system. During the model training process, the learning rate is set to 4e-5, the training batch_size is set to 16, and the verification batch_size is set to 16. , , , The settings are divided into 3, 1, 1, 1.
[0092] Set evaluation indicators. There are three main quantifiable evaluation indicators for query expansion: MAP, NDCG, and Rouge-L. The specific calculation methods of these evaluation indicators are as follows:
[0093] The present invention uses the Mean Average Precision indicator to more carefully consider the ranking position of the relevant documents predicted by the system in the result list. The higher the ranking of the relevant documents, the higher the MAP value. The specific calculation method is as follows:
[0094] ;
[0095] Among them, P@k i is the P@k (Precision@k) indicator of the i-th sample in r, which is mainly used to evaluate the percentage of documents that are truly relevant to the query among the first k results predicted by the model;
[0096] In a given ranking list, let (r) represent the total number of positive samples. Normalized Discounted Cumulative Gain (NDCG) is determined by Discounted Cumulative Gain (DCG) and IdealDiscounted Cumulative Gain (IDCG). First, the calculation of Discounted CumulativeGain (DCG) introduces a logarithmic function mechanism to give greater weight to samples with higher relevance scores in the ranking. Among them, rel(i) represents the relevance between the i-th query and the document in the ranking result. IDCG is the maximum value that can be achieved by arranging from high to low according to the relevance score. Finally, by calculating the ratio of DCG to IDCG, the normalized discounted cumulative gain NDCG@k is obtained. This normalization method enables fair comparison between ranking lists of different lengths, because IDCG always represents the most ideal ranking situation. The formula is as follows:
[0097]
[0098]
[0099] Rouge-L is an evaluation metric commonly used in text generation evaluation tasks. It can measure the degree of overlap between the generated summary and the reference summary. In query expansion, the expanded query results can be regarded as generated content, and the real relevant documents can be regarded as reference summaries, and then the Rouge-L score is calculated. The higher the Rouge-L score, the higher the degree of overlap between the expanded query results and the relevant documents. X is a reference summary sentence (length m), Y is a candidate summary sentence (length n), and LCS (X, Y) is the longest subsequence length of X and Y. The formula is as follows:
[0100]
[0101]
[0102]
[0103] In order to verify the effectiveness of the method proposed in the present invention, the cross-language pre-training models related to the present invention are selected as baseline models, namely mBert, XLM-R, and LaBSE, as follows:
[0104] mBert: As a multilingual pre-trained model, mBert can use multilingual corpora to learn universal language representations. In cross-lingual query expansion, mBert can be used to represent queries, and then use the information in the multilingual corpus to expand the query vocabulary to obtain more relevant documents. By using the universal language representation learned by mBert, the problem of cross-lingual query expansion can be effectively solved.
[0105] XLM-R: Similar to mBert, XLM-R is also a cross-language language model that can better understand and process texts in different languages. In cross-language query expansion, XLM-R can learn richer cross-language language representations by pre-training on corpora in different languages, thereby better expanding queries and improving information retrieval results.
[0106] LaBSE: LaBSE is a language-independent BERT sentence embedding model that can map sentences in different languages into a common language-independent vector space. In cross-language query expansion, LaBSE can be used to represent queries and documents, and then use the semantic information learned from the language-independent sentence representation to expand the query and retrieve related documents.
[0107] Table 1 shows the experimental results of the proposed method compared with the baseline model on the verification machine, as well as Figure 2-Figure 4It can be observed that compared with other baseline models, the model of the present invention has higher query expansion performance on the Chinese-Vietnamese cross-language event and time series validation set. By comparison, the method of the present invention is more effective than the best baseline model mBert, with a Rouge-L score 11% higher than the mBert model, a NDCG@10 score 7% higher than the mBert model, and a MAP score 9.1% higher than the mBert model, verifying that the Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding effectively improves the knowledge expansion ability of the model.
[0108] Table 1
[0109]
[0110] In order to verify the query expansion method proposed in the present invention and the effect of each part in the model, models were designed to conduct experiments.
[0111] According to the innovation of the present invention, Step 2, Step 3 and Step 4 are defined as the following three parts: Event Timing Embedding (ETE), Cross Linguistic Word Embedding (CLE) and Query Extension (QE) are extracted, and three models are designed for ablation experiments, as follows:
[0112] (1) mBert+CLE+ETE: The mBert model is integrated with the cross-language word embedding part and the event and time series embedding part, and the query expansion part is discarded.
[0113] (2) mBert+CLE+QE: The mBert model is integrated with the cross-language word embedding part and the query expansion part, and the event and time series embedding parts are discarded.
[0114] (3) mBert+ETE+QE: The mBert model is integrated with the event and time series embedding part and the query expansion part, and the cross-language word embedding part is discarded.
[0115] The experimental parameters remain the same as before, and the experimental results are shown below.
[0116] Table 2
[0117]
[0118] Table 2 shows the results of the influence of different parts on the model. Figure 5It can be seen that after ablation experiment analysis, in the absence of cross-lingual embedding training, the model's NDCG@10 and MAP scores dropped by 25.06% and 33.54% respectively, which indicates that the model may have insufficient alignment ability in the Chinese-Vietnamese high-level semantic space, which affects its cross-lingual performance and also declines in performance in specific retrieval tasks. When the model lacks event knowledge embedding training, its performance drops by 10.9% and 17.39% in NDCG@10 and MAP scores respectively. This result may indicate that in the absence of an event knowledge attention strategy, the model is difficult to accurately locate the event subject in the query, resulting in low quality of the expanded content, thus affecting the actual retrieval effect. In the absence of a query expansion scoring strategy, the model's performance drops by 15.98% and 24.53% in NDCG@10 and MAP scores respectively. This result reveals that the model still needs to establish an effective query expansion content scoring and selection strategy based on cross-lingual word embedding and event knowledge embedding training.
[0119] The specific implementation modes of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.
Claims
1. A Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding, characterized by: The method comprises: Step 1: Construct a Chinese-Vietnamese cross-language event and time series dataset; Step 2: The time series content corresponding to the event is converted into a vector space through the event and time series word embedding part, and the event knowledge is embedded in it. The embedded event knowledge adopts multilateral measurement method and is embedded by describing multiple aspects of the main event; Step 3: Based on the embedding of events and time-order words, a Chinese-Vietnamese cross-language public dictionary is constructed to expand event knowledge from monolingual to cross-language. By calculating the similarity matrix between Chinese events and event side descriptions, bilingual alignment is achieved in terms of dimensions. Step 4: By calculating the similarity between the query and the expanded content, the query expansion set of the final event subject is obtained; The Step 2 specifically includes the following: Step 2.1, based on the Chinese event subject X and its corresponding Vietnamese event side description term set E, as well as the Vietnamese event temporal content T, obtain multiple description terms in the set E as anchor points in the space; Step 2.2, use the cosine distance D to calculate the distance between each anchor point in the Chinese event subject X and the Vietnamese event side description term set E: ; Step 2.3, finally, find the best position of the Chinese event subject X in the time series vector space, and use the L-BFGS optimization algorithm to minimize the cosine distance D from the anchor point to the event subject X and the mean square error MSE of the distance v between the anchor point and the event subject X after embedding in the time series vector space; The Step 3 specifically includes the following: Based on the original embedding of the Chinese event subject X and the Vietnamese event side description and temporal content (E+T) axes, the corresponding similarity matrices M X and M (E+T) The two axes of correspond to the same words; the linear transformation matrices of the Chinese event subject X and the Vietnamese event side description and temporal content (E+T) are W X and W (E+T) ; Step 3.
1. A dictionary is constructed between the Chinese and Vietnamese languages and encoded into a sparse matrix. In this matrix, if the jth word in the target language is the translation of the ith word in the source language, the value of the element Dij in the sparse matrix is 1, otherwise it is 0. Step 3.2, for M X and M (E+T) Sort the values in each row to get the sorted result sorted(M X ) and sorted(M (E+T) ); Given a word and its sorted(M X ) in the rows, sorted(M (E+T) ) rows are searched using nearest neighbor search to find their corresponding translations; Step 3.3, use the argmax() function to find the best orthogonal mapping that maximizes the similarity of the current dictionary Dic; Step 3.4, calculate the mapping embedding XW X W (E+T) The best dictionary on the similarity matrix of (E+T); Step 3.5, obtain the event subject expansion event set Ce according to the best dictionary obtained by the best orthogonal mapping of the sparse matrix; The Step 4 specifically includes the following: Step 4.1, according to the event knowledge embedding, by calculating the similarity between the query q and the expanded content, the set event Eq of related events is obtained. For the detected related events , create an extended event set Ce; The extended event set Ce contains two types of extended content: (1) extended content that is highly similar to the query q; (2) extended content that is highly similar to the related event e; Step 4.2: Score each expansion event c in the expansion event candidate set Ce using a combination of two different methods: The first method: Calculate the similarity between the extended event c and the query q and the related event e to obtain the similarity between the extended event c and the related event e. , the similarity between the extended event c and the query q , the similarity between related events e and query q , and according to different weight coefficients, obtain the query of the event subject and the expanded event description score ; ; The second method is to add the temporal correlation evaluation index TempRel on the basis of the temporal embedding space. This evaluation method will give a higher score to the related event e of the extended event c occurring within time t. The score is: ; k represents the total number of relevant events e; The final scoring method for each expansion event c is: ; in, , , and is the parameter weight. After obtaining the score of each expansion event c, the expansion events c with high scores are screened out according to the score of each expansion event c and the set threshold, thereby obtaining the final query expansion set.
2. The Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding according to claim 1 is characterized by: In the Step 1, a Chinese-Vietnamese cross-language event and time series data set is constructed, questions in the data set are used to perform Chinese-Vietnamese cross-language queries, and a scoring system is constructed to filter and obtain the Chinese-Vietnamese cross-language event and time series data set.
3. The Chinese-Vietnamese cross-language query expansion method based on event knowledge embedding according to claim 1 is characterized in that: The Step 1 specifically includes: Step 1.1, automatically obtain through web crawler technology; Step 1.2, send a fake web page request through the code, and include the user agent and key information in the request package to successfully establish a secure response with the server of the target page; Step 1.
3. After obtaining the source code of the web page, use the Etree package in Python to parse the specific web page content; next, use the XPath method to collect the anchor coordinates of all required content to collect the specified text in the web page.
