A method and device for semantic analysis of tourism knowledge

By uniformly modeling and aligning heterogeneous tourism knowledge texts, the problem of low semantic understanding accuracy in tourism knowledge semantic analysis is solved, and more efficient tourism knowledge text analysis is achieved, improving user experience .

CN114036956BActive Publication Date: 2024-12-03TSINGHUA UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111371152.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-18
Publication Date
2024-12-03
Estimated Expiration
2041-11-18

AI Technical Summary

Technical Problem

The existing pre-trained language models have poor results in semantic analysis of tourism knowledge, and lack the learning of document structure and structured knowledge, resulting in low accuracy in semantic understanding.

Method used

Design a tourism knowledge semantic analysis method, build a corpus by obtaining heterogeneous multi-format text, aligning unstructured text, semi-structured text, and well-structured knowledge triplet text, and modeling uniformly into the same context representation space , using unsupervised training tasks and TF-IDF similarity analysis principles, a pre-trained language model based on heterogeneous tourism knowledge is constructed.

Benefits of technology

It improves the accuracy and robustness of text semantic understanding of tourism knowledge, improves user experience, and enhances the analysis ability of texts of different granularity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114036956B_ABST
    Figure CN114036956B_ABST
Patent Text Reader

Abstract

The present invention provides a method and device for semantic analysis of tourism knowledge. The method comprises: obtaining a tourism knowledge text to be understood; inputting the tourism knowledge text to be understood into a pre-trained language model based on heterogeneous tourism knowledge to obtain a semantic understanding result output by the pre-trained language model; the pre-trained language model is obtained by using preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly modeling the training samples into the same context representation space based on corresponding unsupervised training tasks. The tourism knowledge semantic analysis method provided by the present invention can analyze tourism knowledge texts of different granularities by using a pre-trained language model based on heterogeneous tourism knowledge obtained by training with texts in multiple formats, so as to improve the accuracy and robustness of semantic understanding of tourism knowledge texts, thereby enhancing the user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data model analysis, and in particular to a method and device for semantic analysis of tourism knowledge. In addition, the present invention also relates to an electronic device and a processor-readable storage medium. Background Art

[0002] The pre-trained language model (Bidirectional Encoder Representation from Transformers; BERT) is a pre-trained language representation model, and its extension is intended to enhance the existing language model in order to better understand the downstream text. At present, the pre-trained language models in the existing technology are designed with different learning tasks, different semantic granularity, different model architectures, and different learning algorithms, but they are usually pre-trained using text in a single format, lacking the learning of document structure and related structured knowledge, resulting in poor results in semantic analysis of knowledge in specific tourism fields. However, in practice, there is usually a large amount of unused semi-structured and well-structured text data in tourism data documents, which is crucial for the hierarchical understanding of words, entities, and paragraphs. Therefore, how to design an efficient and stable tourism knowledge semantic analysis solution to improve the accuracy of semantic understanding of tourism knowledge text has become a problem that needs to be solved urgently. Summary of the invention

[0003] To this end, the present invention provides a tourism knowledge semantic analysis method and device to solve the defect that the tourism knowledge semantic analysis solution based on the traditional pre-trained language model in the prior art has high limitations, resulting in poor accuracy in semantic understanding of tourism knowledge text.

[0004] In a first aspect, the present invention provides a method for semantic analysis of tourism knowledge, comprising: obtaining a tourism knowledge text to be understood; inputting the tourism knowledge text to be understood into a pre-trained language model based on heterogeneous tourism knowledge, and obtaining a semantic understanding result output by the pre-trained language model; wherein the pre-trained language model uses preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly models the training samples into the same context representation space based on corresponding unsupervised training tasks.

[0005] Furthermore, the tourism knowledge semantic analysis method further includes: determining the pre-trained language model;

[0006] The determining of the pre-trained language model specifically includes:

[0007] Obtaining heterogeneous multi-format texts and constructing a corpus; the corpus includes the unstructured text, the semi-structured text, and preset knowledge triple texts that meet structural conditions;

[0008] Aligning the unstructured text with the preset knowledge triple text to determine the corresponding relationship between the heterogeneous multi-format texts;

[0009] Based on the preset corresponding objective function, unsupervised training is performed on the unstructured text, the semi-structured text, and the preset knowledge triple text respectively, and the unstructured text, the semi-structured text, and the preset knowledge triple text are uniformly modeled into the same context representation space to obtain the pre-trained language model.

[0010] Furthermore, aligning the unstructured text with the preset knowledge triple text specifically includes:

[0011] Based on the preset TF-IDF similarity analysis principle, the corresponding TF-IDF vector is used to determine the similarity between each preset knowledge triple text and the unstructured text, and the unstructured text is aligned with the preset knowledge triple text based on the similarity result.

[0012] Furthermore, the pre-trained language model is used to perform semantic understanding on the tourism knowledge text to be understood based on the representation features obtained by extracting features from the heterogeneous multi-format text.

[0013] Furthermore, the tourism knowledge text to be understood includes tourism knowledge text of at least one granularity of word text, entity text, and subject text.

[0014] In a second aspect, the present invention further provides a tourism knowledge semantic analysis device, comprising:

[0015] A tourism knowledge text acquisition unit, used for acquiring the tourism knowledge text to be understood;

[0016] The tourism knowledge semantic analysis unit is used to input the tourism knowledge text to be understood into a pre-trained language model based on heterogeneous tourism knowledge to obtain a semantic understanding result output by the pre-trained language model; wherein the pre-trained language model uses preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly models the training samples into the same context representation space based on corresponding unsupervised training tasks.

[0017] Furthermore, the tourism knowledge semantic analysis device further includes: a model training unit, used to determine the pre-trained language model;

[0018] The determining of the pre-trained language model specifically includes:

[0019] Obtaining heterogeneous multi-format texts and constructing a corpus; the corpus includes the unstructured text, the semi-structured text, and preset knowledge triple texts that meet structural conditions;

[0020] Aligning the unstructured text with the preset knowledge triple text to determine the corresponding relationship between the heterogeneous multi-format texts;

[0021] Based on the preset corresponding objective function, unsupervised training is performed on the unstructured text, the semi-structured text, and the preset knowledge triple text respectively, and the unstructured text, the semi-structured text, and the preset knowledge triple text are uniformly modeled into the same context representation space to obtain the pre-trained language model.

[0022] Furthermore, the pre-trained language model is used to perform semantic understanding on the tourism knowledge text to be understood based on the representation features obtained by extracting features from the heterogeneous multi-format text.

[0023] Furthermore, the unstructured text is aligned with the preset knowledge triple text, specifically including: based on the preset TF-IDF similarity analysis principle, using the corresponding TF-IDF vector to determine the similarity between each preset knowledge triple text and the unstructured text, and aligning the unstructured text with the preset knowledge triple text based on the similarity result.

[0024] Furthermore, the tourism knowledge text to be understood includes tourism knowledge text of at least one granularity of word text, entity text, and subject text.

[0025] In a third aspect, the present invention further provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the tourism knowledge semantic analysis method as described in any one of the above items are implemented.

[0026] In a fourth aspect, the present invention further provides a processor-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the tourism knowledge semantic analysis method as described in any one of the above items are implemented.

[0027] The tourism knowledge semantic analysis method provided by the present invention can analyze tourism knowledge texts of different granularities by using a pre-trained language model based on heterogeneous tourism knowledge obtained by training texts in various formats, so as to improve the accuracy and robustness of the semantic understanding of the tourism knowledge texts, thereby enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 It is a flow chart of a method for semantic analysis of tourism knowledge provided by an embodiment of the present invention;

[0030] Figure 2 It is a schematic diagram of unsupervised pre-training in the tourism knowledge semantic analysis method provided by an embodiment of the present invention;

[0031] Figure 3 is a schematic diagram of the training process of the pre-trained language model provided by an embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of the structure of a tourism knowledge semantic analysis device provided by an embodiment of the present invention;

[0033] Figure 5 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] The following is a detailed description of an embodiment of the tourism knowledge semantic analysis method according to the present invention. Figure 1 As shown, it is a flow chart of the tourism knowledge semantic analysis method provided by an embodiment of the present invention, and the specific implementation process includes the following steps:

[0036] Step 101: Obtain tourism knowledge text to be understood.

[0037] In the embodiment of the present invention, before executing this step, model training needs to be performed in advance to obtain a pre-trained language model that meets the preset conditions of the present invention. Specifically, a heuristic method is first used to align multi-format texts, then an unsupervised training task is designed, and finally the knowledge-enhanced pre-trained language model is applied to different downstream tasks in the tourism field.

[0038] In the specific implementation process, determining the pre-trained language model includes: obtaining heterogeneous multi-format texts and constructing a corpus; the corpus contains the unstructured text, the semi-structured text and the preset knowledge triple text that meets the structural conditions; aligning the unstructured text with the preset knowledge triple text to determine the corresponding relationship between the heterogeneous multi-format texts; based on the preset corresponding objective function, unsupervised training is performed on the unstructured text, the semi-structured text and the preset knowledge triple text respectively, and the unstructured text, the semi-structured text and the preset knowledge triple text are uniformly modeled into the same context representation space to obtain a pre-trained language model (Travel BERT) based on heterogeneous tourism knowledge. The pre-trained language model is used to perform semantic understanding on the tourism knowledge text to be understood based on the representation features obtained by feature extraction of the heterogeneous multi-format texts. Among them, the unstructured text is aligned with the preset knowledge triple text, and the specific implementation process includes: based on the preset TF-IDF (Term Frequency-Inverse Document Frequency) similarity analysis principle, using the corresponding TF-IDF vector to determine the similarity between each preset knowledge triple text and the unstructured text, and aligning the unstructured text with the preset knowledge triple text based on the similarity result.

[0039] like Figure 3 As shown, in the model training process of the embodiment of the present invention, when aligning multi-format texts, in order to reduce the operational pressure of the model and retain sufficient unstructured text, the present invention adopts a knowledge triple retrieval strategy to correspond the unstructured text to the relevant preset knowledge triple text. The above alignment process is performed based on TF-IDF similarity, so that the knowledge triple text contained in the unstructured text description can be found. Here, the TF-IDF vector is used to calculate the similarity between each preset knowledge triple text and the unstructured text, as shown below.

[0040] similarity=cos(vec(s (e) ),vec((s,p,o)))

[0041] Among them, vec(.) is the mapping operation that converts unstructured text into TF-IDF vector; s (e) and (s,p,o) are considered as different unstructured texts. Thus, the cosine similarity between unstructured texts can be calculated. This method can be used to calculate the TF-IDF of a corpus consisting of tourist attraction text paragraphs and knowledge triple texts obtained from various encyclopedia websites, as shown below.

[0042]

[0043] where d is the document containing the term t; f t,d is the term frequency of t in document d; N is the number of documents in the corpus; n t is the number of documents containing t.

[0044] During the unsupervised training process, as Figure 2 (a) shows, the original data consists of the unstructured text "A building is a two-generation royal palace in the target area, formerly known as B City, located at the center of the central axis of the target area", the corresponding title "Abstract", and the relevant knowledge triples <A building, location, target area>, <A building, era, XXXX>, <A building, former name, B City>. As Figure 2 (b) the lower part shows, the input consists of the text "A building is a two-generation royal palace in the target area, formerly known as B City, located at the center of the central axis of the target area", the title "Abstract", the positive sample triples PT, and the negative sample triples NT. The following formula is used to calculate the representation of each element:

[0045] h (D) ,h (T) ,h (KG) = F bert (<[CLS]; s (e) ; [SEP0]; t (e) ; [SEP1]; (e,p j ,o i ) ; [SEP2]…>)

[0046] where the new symbol [SEP0] is used to identify the title; [SEPi] (i>1) is used to represent each knowledge triple. Each element of the triple is regarded as text rather than an identifier, and the advantage is that different forms of knowledge or text can be represented in the same context representation space. The model needs to predict whether the title of the subsection matches the paragraph text, and whether the predicate of the triple has been modified. At the same time, the masked language model loss is also retained during the training process to help the model learn token-level knowledge or text from the text in the tourism domain.

[0047] In the model training process of the embodiment of the present invention, entity knowledge is learned through preset knowledge triple texts (well-structured texts) that meet structural conditions. Specifically, tokens can be randomly masked with a probability of 15% during the pre-training process. In order to better learn the attributes of entities, in addition to shielding paragraph texts, the pre-trained language model can also randomly shield knowledge triple texts. Among them, an entity may have several to dozens of attributes, such as address, location, scenic spots, climate type, scenic spot level, scenic spot type, construction time, completion time, etc. The triple text classification task is used to enhance entity knowledge learning. For example, a preset program can be used to add noise to the triple text, and then the pre-trained language model is allowed to determine whether the triple text has been modified, so that the pre-trained language model can verify the knowledge triple text through the self-attention mechanism and unstructured text. Among them, the method of adding noise adopted can be an attribute resampling operation, the purpose of which is to randomly replace the attributes p of some triple texts with the attributes of other triple texts. As shown below.

[0048]

[0049] Where P represents all attributes. Resampling attributes can improve the pre-trained language model’s understanding of relational semantics.

[0050] In addition, paragraph knowledge can also be learned through semi-structured text.

[0051] In the unsupervised training scenario, the present invention uses the section title as a natural label for the corresponding paragraph to guide the pre-trained language model to understand the topic of the paragraph. Specifically, a preset program can be used to automatically construct a "title-paragraph" pair, and the pre-trained language model only needs to predict whether the title has been modified based on the paragraph.

[0052] For multi-level titles, the title closest to the paragraph is used to obtain a more specific topic. When pre-training the language model, the present invention connects the text with the corresponding title, which is represented as <[CLS];s (e) ;[SEP0];t (e) ;…>. The present invention uses the representation of [SEP0] for classification. To generate negative samples, for the title t of "Abstract", the program will sample another section title in the same article to replace the original title as shown below.

[0053]

[0054] Among them, t represents the title "Abstract"; Indicates the section title; T indicates the preset time parameter.

[0055] In the model training process of the embodiment of the present invention, for the three pre-trained subtasks, the present invention can adopt the following corresponding objective functions.

[0056]

[0057] in, Denote the objective functions corresponding to the three subtasks respectively; |D| is the size of the data set; Θ is the corresponding model parameter; λ and μ are hyperparameters used to measure the influence of each subtask. The training loss is the sum of the deviation of the cloze task, the deviation of the triple classification task, and the deviation of the title matching task. The present invention adopts negative log-likelihood as the optimization target, as shown below:

[0058]

[0059] Among them, p (mlm) Represents the probability of the true category of the cloze task, and i represents the index of the token.

[0060]

[0061] Among them, p (tc) represents the probability of the true category of the triple classification task; i represents the index of the triple.

[0062]

[0063] Among them, p (tmt) Represents the probability of the true category of the title matching task. Finally, the AdamW optimization algorithm is used to train the model parameters.

[0064] In addition, the specific implementation process also includes fine-tuning the pre-trained language model (Travel BERT) based on heterogeneous tourism knowledge for tourism NLP (natural language processing) tasks.

[0065] Specifically, the pre-trained language model based on heterogeneous tourism knowledge can be evaluated using five preset downstream tourism NLP tasks, including named entity recognition, open information extraction, question answering, fine-grained entity type labeling, and dialogue. When fine-tuning the downstream tourism NLP tasks, the pre-trained language model does not need to change the input text because it can learn heterogeneous knowledge and thus learn better parameters during the pre-training stage. Then the pre-trained language model uses the learned knowledge (parameters) to better solve the downstream tourism NLP tasks.

[0066] For the named entity recognition task, the present invention adopts a sequence tagging form and uses the vector of each token in the last layer of the pre-trained language model to classify the entity label. For the fine-grained entity type labeling task, in order to assign fine-grained type labels to entity mentions in the text. The present invention adds two special symbols [ENT] to indicate the position of the entity and uses the [CLS] vector of the last layer of the pre-trained language model to classify the label. For the open information extraction task, the present invention uses a two-stage text fragment extraction reading comprehension model. Specifically, the present invention first trains a relation prediction model to extract multiple predicate fragments in a sentence. The pre-trained language model extracts each fragment in a way that predicts the start and end positions. In an embodiment of the present invention, a threshold can be used to select multiple predicate fragments because a sentence may contain multiple triple texts (knowledge triple texts). By adding two special symbols [REL] to indicate the position of the predicate fragment, the present invention then trains an entity prediction model to extract the subject and object fragments of each predicate. For the question-answering task, the present invention uses the [CLS] vector of the last layer of the pre-trained language model to calculate and rank the matching score of each candidate answer to the question. For the dialogue task, the present invention adopts a retrieval-based strategy. The training task is to predict whether the candidate response is the correct next sentence based on the context. For the test process, the candidate response with the highest probability is selected during the specific implementation process.

[0067] Step 102: Input the tourism knowledge text to be understood into a pre-trained language model based on heterogeneous tourism knowledge to obtain a semantic understanding result output by the pre-trained language model; wherein the pre-trained language model uses preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly models the training samples into the same context representation space based on corresponding unsupervised training tasks.

[0068] In an embodiment of the present invention, after obtaining a pre-trained language model based on heterogeneous tourism knowledge that meets the application requirements through the above-mentioned model training method, the acquired tourism knowledge text to be understood can be input into the pre-trained language model based on heterogeneous tourism knowledge, thereby obtaining a semantic understanding result output by the pre-trained language model. The tourism knowledge text to be understood includes tourism knowledge text of at least one granularity among word text, entity text, theme text, etc. The present invention provides a pre-trained language model based on heterogeneous tourism knowledge used in a tourism knowledge semantic analysis method, which simultaneously models unstructured text, semi-structured text, and well-structured text into the same context representation space through a pre-trained language model.

[0069] The tourism knowledge semantic analysis method described in the embodiment of the present invention can analyze tourism knowledge texts of different granularities by using a pre-trained language model based on heterogeneous tourism knowledge obtained by training texts in various formats, so as to improve the accuracy and robustness of the semantic understanding of tourism knowledge texts, and can effectively improve the analysis effect when processing downstream NLP tasks, thereby improving the user experience.

[0070] Corresponding to the above-mentioned tourism knowledge semantic analysis method, the present invention also provides a tourism knowledge semantic analysis device. Since the embodiment of the device is similar to the above-mentioned method embodiment, the description is relatively simple. For relevant parts, please refer to the description of the above-mentioned method embodiment. The embodiment of the tourism knowledge semantic analysis device described below is only illustrative. Please refer to Figure 4 As shown, it is a structural schematic diagram of a tourism knowledge semantic analysis device provided by an embodiment of the present invention.

[0071] The tourism knowledge semantic analysis device of the present invention specifically includes the following parts:

[0072] A tourism knowledge text acquisition unit 401 is used to acquire the tourism knowledge text to be understood;

[0073] The tourism knowledge semantic analysis unit 402 is used to input the tourism knowledge text to be understood into a pre-trained language model based on heterogeneous tourism knowledge to obtain a semantic understanding result output by the pre-trained language model; wherein the pre-trained language model uses preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly models the training samples into the same context representation space based on corresponding unsupervised training tasks.

[0074] Furthermore, the tourism knowledge semantic analysis device further includes: a model training unit, used to determine the pre-trained language model;

[0075] The determining of the pre-trained language model specifically includes:

[0076] Obtaining heterogeneous multi-format texts and constructing a corpus; the corpus includes the unstructured text, the semi-structured text, and preset knowledge triple texts that meet structural conditions;

[0077] Aligning the unstructured text with the preset knowledge triple text to determine the corresponding relationship between the heterogeneous multi-format texts;

[0078] Based on the preset corresponding objective function, unsupervised training is performed on the unstructured text, the semi-structured text, and the preset knowledge triple text respectively, and the unstructured text, the semi-structured text, and the preset knowledge triple text are uniformly modeled into the same context representation space to obtain the pre-trained language model.

[0079] Furthermore, the pre-trained language model is used to perform semantic understanding on the tourism knowledge text to be understood based on the representation features obtained by extracting features from the heterogeneous multi-format text.

[0080] Furthermore, the unstructured text is aligned with the preset knowledge triple text, specifically including: based on the preset TF-IDF similarity analysis principle, using the corresponding TF-IDF vector to determine the similarity between each preset knowledge triple text and the unstructured text, and aligning the unstructured text with the preset knowledge triple text based on the similarity result.

[0081] Furthermore, the tourism knowledge text to be understood includes tourism knowledge text of at least one granularity of word text, entity text, and subject text.

[0082] The tourism knowledge semantic analysis device described in the embodiment of the present invention can analyze tourism knowledge texts of different granularities by using a pre-trained language model based on heterogeneous tourism knowledge obtained by training texts in various formats, so as to improve the accuracy and robustness of the semantic understanding of the tourism knowledge texts, thereby enhancing the user experience.

[0083] Corresponding to the tourism knowledge semantic analysis method provided above, the present invention also provides an electronic device. Since the embodiment of the electronic device is similar to the embodiment of the method described above, the description is relatively simple. For the relevant parts, please refer to the description of the embodiment of the method described above. The electronic device described below is only for illustration. Figure 5As shown, it is a schematic diagram of the physical structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may include: a processor 501, a memory 502 and a communication bus 503, wherein the processor 501 and the memory 502 communicate with each other through the communication bus 503, and communicate with the outside through the communication interface 504. The processor 501 can call the logic instructions in the memory 502 to execute the tourism knowledge semantic analysis method, which includes: obtaining a tourism knowledge text to be understood; inputting the tourism knowledge text to be understood into a pre-trained language model based on heterogeneous tourism knowledge, and obtaining a semantic understanding result output by the pre-trained language model; wherein the pre-trained language model is obtained by using preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly modeling the training samples into the same context representation space based on the corresponding unsupervised training task.

[0084] In addition, the logic instructions in the above-mentioned memory 502 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as storage chips, USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks.

[0085] On the other hand, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program stored on a processor-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the tourism knowledge semantic analysis method provided by the above-mentioned method embodiments. The method includes: obtaining a tourism knowledge text to be understood; inputting the tourism knowledge text to be understood into a pre-trained language model based on heterogeneous tourism knowledge, and obtaining a semantic understanding result output by the pre-trained language model; wherein the pre-trained language model uses preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly models the training samples into the same context representation space based on the corresponding unsupervised training task.

[0086] On the other hand, an embodiment of the present invention further provides a processor-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor, it is implemented to execute the tourism knowledge semantic analysis method provided by the above embodiments. The method includes: obtaining a tourism knowledge text to be understood; inputting the tourism knowledge text to be understood into a pre-trained language model based on heterogeneous tourism knowledge, and obtaining a semantic understanding result output by the pre-trained language model; wherein the pre-trained language model uses preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly models the training samples into the same context representation space based on the corresponding unsupervised training task.

[0087] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0088] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0089] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tourism knowledge semantic analysis method, characterized in that: include: Obtain tourism knowledge texts to be understood; The tourism knowledge text to be understood is input into a pre-trained language model based on heterogeneous tourism knowledge to obtain a semantic understanding result output by the pre-trained language model; wherein the pre-trained language model is obtained by using preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly modeling the training samples into the same context representation space based on corresponding unsupervised training tasks; Before obtaining the tourism knowledge text to be understood, it also includes: Obtaining heterogeneous multi-format texts and constructing a corpus; the corpus includes the unstructured text, the semi-structured text, and preset knowledge triple texts that meet structural conditions; The unstructured text is aligned with the preset knowledge triple text to determine the corresponding relationship between the heterogeneous multi-format texts; the unstructured text is aligned with the preset knowledge triple text, specifically including: based on the preset TF-IDF similarity analysis principle, using the corresponding TF-IDF vector to determine the similarity between each preset knowledge triple text and the unstructured text, and aligning the unstructured text with the preset knowledge triple text based on the similarity result; specifically, the alignment process is performed based on TF-IDF similarity to find the knowledge triple text contained in the unstructured text description; the TF-IDF vector is used to calculate the similarity between each preset knowledge triple text and the unstructured text, as shown below: similarity=cos(vec(s (e) ),vec((s,p,o))) Among them, vec(.) is the mapping operation that converts unstructured text into TF-IDF vector; s (e) and (s, p, o) are different unstructured texts; the cosine similarity between the unstructured texts is calculated; the TF-IDF of the corpus consisting of tourist attraction text paragraphs and knowledge triple texts obtained from various encyclopedia websites is calculated as follows: where d is the document containing term t; f t,d is the frequency of word t in document d; N is the number of documents in the corpus; n t is the number of documents containing t; Based on the preset corresponding objective function, unsupervised training is performed on the unstructured text, the semi-structured text, and the preset knowledge triple text respectively, and the unstructured text, the semi-structured text, and the preset knowledge triple text are uniformly modeled into the same context representation space to obtain the pre-trained language model; Among them, during the model training process, the following corresponding objective functions are adopted: in, They represent the objective functions corresponding to the three subtasks respectively; |D| is the size of the dataset; Θ is the corresponding model parameter; λ and μ are hyperparameters used to measure the influence of each subtask; the training loss is the sum of the deviation of the cloze task, the deviation of the triple classification task, and the deviation of the title matching task.

2. The tourism knowledge semantic analysis method according to claim 1 is characterized in that: The pre-trained language model is used to perform semantic understanding on the tourism knowledge text to be understood based on the representation features obtained by extracting features from the heterogeneous multi-format text.

3. The tourism knowledge semantic analysis method according to claim 1 is characterized in that: The tourism knowledge text to be understood includes tourism knowledge text of at least one granularity among word text, entity text, and subject text.

4. A tourism knowledge semantic analysis device, characterized in that: include: A tourism knowledge text acquisition unit, used for acquiring the tourism knowledge text to be understood; A tourism knowledge semantic analysis unit, used for inputting the tourism knowledge text to be understood into a pre-trained language model based on heterogeneous tourism knowledge, and obtaining a semantic understanding result output by the pre-trained language model; wherein the pre-trained language model is obtained by using preset unstructured text, semi-structured text and preset knowledge triple text that meets structural conditions as training samples, and uniformly modeling the training samples into the same context representation space based on corresponding unsupervised training tasks; The invention also includes: a model training unit; the model training unit is used to: Obtaining heterogeneous multi-format texts and constructing a corpus; the corpus includes the unstructured text, the semi-structured text, and preset knowledge triple texts that meet structural conditions; The unstructured text is aligned with the preset knowledge triple text to determine the corresponding relationship between the heterogeneous multi-format texts; the unstructured text is aligned with the preset knowledge triple text, specifically including: based on the preset TF-IDF similarity analysis principle, using the corresponding TF-IDF vector to determine the similarity between each preset knowledge triple text and the unstructured text, and aligning the unstructured text with the preset knowledge triple text based on the similarity result; specifically, the alignment process is performed based on TF-IDF similarity to find the knowledge triple text contained in the unstructured text description; the TF-IDF vector is used to calculate the similarity between each preset knowledge triple text and the unstructured text, as shown below: similarity=cos(vec(s (e) ),vec((s,p,o))) Among them, vec(.) is the mapping operation that converts unstructured text into TF-IDF vector; s (e) and (s, p, o) are different unstructured texts; the cosine similarity between the unstructured texts is calculated; the TF-IDF of the corpus consisting of tourist attraction text paragraphs and knowledge triple texts obtained from various encyclopedia websites is calculated as follows: where d is the document containing term t; f t,d is the frequency of word t in document d; N is the number of documents in the corpus; n t is the number of documents containing t; Based on the preset corresponding objective function, unsupervised training is performed on the unstructured text, the semi-structured text, and the preset knowledge triple text respectively, and the unstructured text, the semi-structured text, and the preset knowledge triple text are uniformly modeled into the same context representation space to obtain the pre-trained language model; Among them, during the model training process, the following corresponding objective functions are adopted: in, They represent the objective functions corresponding to the three subtasks respectively; |D| is the size of the dataset; Θ is the corresponding model parameter; λ and μ are hyperparameters used to measure the influence of each subtask; the training loss is the sum of the deviation of the cloze task, the deviation of the triple classification task, and the deviation of the title matching task.

5. The tourism knowledge semantic analysis device according to claim 4, characterized in that: The pre-trained language model is used to perform semantic understanding on the tourism knowledge text to be understood based on the representation features obtained by extracting features from the heterogeneous multi-format text.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the tourism knowledge semantic analysis method as claimed in any one of claims 1 to 3 are implemented.

7. A processor-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the tourism knowledge semantic analysis method as described in any one of claims 1 to 3 are implemented.