Multi-element Tibetan data feature fusion method and system, storage medium and electronic equipment

Abstract

The invention provides a multivariate Tibetan data feature fusion method and system, a storage medium and electronic equipment, and relates to the field of natural language processing. In the method, for each Tibetan syllable, a Tibetan vocabulary dictionary is adopted for matching so as to extract a potential Tibetan vocabulary list; and sequentially splicing the Tibetan syllable sequence and the potential Tibetan vocabulary list, and carrying out serial number labeling on a starting position and an ending position on each element in the Tibetan syllable sequence and the potential Tibetan vocabulary list to calculate a relative position relationship between any two elements so as to obtain a corresponding relative position code. Further, for each Tibetan syllable, a Tibetan syllable component dictionary is adopted for matching so as to extract corresponding component features, and Tibetan syllable component feature information is obtained after embedding operation; and fusing the relative position codes and Tibetan syllable component feature information to obtain multivariate Tibetan data features. According to the embodiment of the invention, Tibetan syllables, vocabularies and syllable component information are effectively fused, and the effect of a Tibetan named entity recognition task can be greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a multi-dimensional Tibetan data feature fusion method, system, storage medium and electronic device. Background Art

[0002] At present, Tibetan named entity recognition technology is relatively backward in the field of intelligent information processing, especially compared with mainstream languages ​​such as English and Chinese. In fact, research in this field is not only of great academic significance, but also has extensive value in practical applications.

[0003] In-depth research and application of Tibetan named entity recognition technology can promote the informatization of Tibetan language and help the digital transformation of Tibetan language in multiple fields such as language, culture and economy. Especially in the fields of cross-language information processing, machine translation and speech recognition, the research and application of Tibetan named entity recognition technology has practical significance and social value.

[0004] In the related technology, the existing Tibetan named entity recognition technology is mostly based on the Tibetan syllable embedding method, ignoring the Tibetan vocabulary information and Tibetan syllable component information. This method is often difficult to achieve good results when dealing with Tibetan named entity recognition tasks. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In view of the deficiencies of the prior art, the present invention provides a multi-layer Tibetan data feature fusion method, system, storage medium and electronic device, which solves the technical problem of ignoring Tibetan vocabulary information and Tibetan syllable component information.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A multi-dimensional Tibetan data feature fusion method, comprising:

[0010] For each Tibetan syllable of the Tibetan syllable sequence to be subjected to entity recognition, a pre-built Tibetan vocabulary dictionary is used for matching to extract a potential Tibetan vocabulary list;

[0011] Sequentially splicing the Tibetan syllable sequence and the potential Tibetan vocabulary list, and marking the start position and end position of each element of the spliced ​​sequence text;

[0012] The starting position and ending position numbers of each syllable element are both the position index of the syllable element in the Tibetan syllable sequence; the starting position number of each Tibetan vocabulary element is the position index of the first Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence, and the ending position number is the position index of the last Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence;

[0013] Based on the numbers of the starting and ending positions of each element, calculate the relative position relationship between any two elements;

[0014] Based on the relative position relationship between every two elements, the corresponding relative position code is obtained.

[0015] Preferably, it also includes:

[0016] For each Tibetan syllable in the Tibetan syllable sequence to be subject to entity recognition, a pre-constructed Tibetan syllable component dictionary is used for matching to extract component features of each Tibetan syllable;

[0017] Embedding the component features of each Tibetan syllable to obtain Tibetan syllable component feature information;

[0018] The relative position coding and the Tibetan syllable component feature information are integrated to obtain multi-dimensional Tibetan data features integrating Tibetan syllables, Tibetan vocabulary and Tibetan syllable components.

[0019] Preferably, it also includes:

[0020] The multivariate Tibetan data features are used as input to a pre-trained model to perform a Tibetan named entity recognition task.

[0021] Preferably, the calculation of the relative position relationship between any two elements based on the numbers of the starting position and the ending position of each element comprises:

[0022]

[0023] Among them, start[i] and end[i] represent the elements X respectively. i The starting and ending position numbers of i is the i-th element in the concatenation result;

[0024] start[j] and end[j] represent the elements X j The starting and ending position numbers of j is the jth element in the concatenation result;

[0025] Represents element X i and X jThe distance from the starting position;

[0026] Represents element X i The starting position of the element X j The distance to the end position;

[0027] Represents element X i The end position of the element X j The distance from the starting position;

[0028] Represents element X i The end position of the element X j The distance to the end position.

[0029] A multi-dimensional Tibetan data feature fusion system, comprising:

[0030] A vocabulary extraction module, for matching each Tibetan syllable of the Tibetan syllable sequence to be subject to entity recognition with a pre-built Tibetan vocabulary dictionary to extract a potential Tibetan vocabulary list;

[0031] A numbering module is used to sequentially splice the Tibetan syllable sequence and the potential Tibetan vocabulary list, and to mark the start position and end position of each element of the spliced ​​sequence text;

[0032] The starting position and ending position numbers of each syllable element are both the position index of the syllable element in the Tibetan syllable sequence; the starting position number of each Tibetan vocabulary element is the position index of the first Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence, and the ending position number is the position index of the last Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence;

[0033] A relationship calculation module, used to calculate the relative position relationship between any two elements based on the numbers of the starting position and the ending position of each element;

[0034] The position coding module is used to obtain the corresponding relative position coding based on the relative position relationship between every two elements.

[0035] Preferably, it also includes:

[0036] A component extraction module is used to match each Tibetan syllable in the Tibetan syllable sequence to be subject to entity recognition with a pre-built Tibetan syllable component dictionary to extract component features of each Tibetan syllable;

[0037] A component encoding module is used to embed the component features of each Tibetan syllable to obtain the component feature information of the Tibetan syllable;

[0038] The feature fusion module is used to fuse the relative position code and the Tibetan syllable component feature information to obtain multi-dimensional Tibetan data features that fuse Tibetan syllables, Tibetan words and Tibetan syllable components.

[0039] Preferably, it also includes:

[0040] The entity recognition module is used to use the multivariate Tibetan data features as input to a pre-trained model to perform a Tibetan named entity recognition task.

[0041] Preferably, the calculation of the relative position relationship between any two elements based on the numbers of the starting position and the ending position of each element comprises:

[0042]

[0043] Among them, start[i] and end[i] represent the elements X respectively. i The starting and ending position numbers of i is the i-th element in the concatenation result;

[0044] start[j] and end[j] represent the elements X j The starting and ending position numbers of j is the jth element in the concatenation result;

[0045] Represents element X i and X j The distance from the starting position;

[0046] Represents element X i The starting position of the element X j The distance to the end position;

[0047] Represents element X i The end position of the element X j The distance from the starting position;

[0048] Represents element X i The end position of the element X j The distance to the end position.

[0049] A storage medium stores a computer program for multi-dimensional Tibetan data feature fusion, wherein the computer program enables a computer to execute the multi-dimensional Tibetan data feature fusion method as described above.

[0050] An electronic device, comprising:

[0051] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the multivariate Tibetan data feature fusion method as described above.

[0052] (III) Beneficial effects

[0053] The present invention provides a multi-dimensional Tibetan data feature fusion method, system, storage medium and electronic device. Compared with the prior art, it has the following beneficial effects:

[0054] In the present invention, for each Tibetan syllable of the Tibetan syllable sequence to be subject to entity recognition, a pre-constructed Tibetan vocabulary dictionary is used for matching to extract a potential Tibetan vocabulary list; the Tibetan syllable sequence and the potential Tibetan vocabulary list are sequentially spliced, and each element of the spliced ​​sequence text is numbered and marked with a start position and an end position; based on the numbers of the start position and the end position of each element, the relative position relationship between any two elements is calculated; based on the relative position relationship between every two elements, the corresponding relative position code is obtained. The present invention at least integrates Tibetan syllables and Tibetan vocabulary information, and can improve the effect of the Tibetan named entity recognition task to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A block diagram of a first multi-dimensional Tibetan data feature fusion method provided by an embodiment of the present invention;

[0057] Figure 2 A block diagram of a second multi-dimensional Tibetan data feature fusion method provided by an embodiment of the present invention;

[0058] Figure 3 A block diagram of a third multi-dimensional Tibetan data feature fusion method provided by an embodiment of the present invention;

[0059] Figure 4 A schematic diagram of a planar network structure integrating vocabulary information provided by an embodiment of the present invention;

[0060] Figure 5 A schematic diagram of the characteristics of the components of a Tibetan syllable provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] The embodiments of the present application solve the technical problem of ignoring Tibetan vocabulary information and Tibetan syllable component information by providing a multi-dimensional Tibetan data feature fusion method, system, storage medium and electronic device.

[0063] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0064] In the embodiment of the present invention, for each Tibetan syllable in the Tibetan syllable sequence to be recognized as an entity, a pre-constructed Tibetan vocabulary dictionary is used for matching to extract a potential Tibetan vocabulary list; the Tibetan syllable sequence and the potential Tibetan vocabulary list are sequentially spliced, and each element of the spliced ​​sequence text is numbered with the start position and the end position; based on the numbers of the start position and the end position of each element, the relative position relationship between any two elements is calculated; based on the relative position relationship between every two elements, the corresponding relative position code is obtained. The embodiment of the present invention at least integrates Tibetan syllables and Tibetan vocabulary information, which can improve the effect of the Tibetan named entity recognition task to a certain extent.

[0065] Furthermore, the embodiment of the present invention includes: for each Tibetan syllable of the Tibetan syllable sequence to be subject to entity recognition, a pre-constructed Tibetan syllable component dictionary is used for matching to extract the component features of each Tibetan syllable; the component features of each Tibetan syllable are embedded to obtain Tibetan syllable component feature information; the relative position coding and Tibetan syllable component feature information are integrated to obtain multi-dimensional Tibetan data features integrating Tibetan syllables, Tibetan vocabulary and Tibetan syllable components. The embodiment of the present invention effectively integrates Tibetan syllables, vocabulary and syllable component information, which can greatly improve the effect of Tibetan named entity recognition tasks.

[0066] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0067] Embodiment 1:

[0068] like Figure 1 As shown, the embodiment of the present invention provides a multi-dimensional Tibetan data feature fusion method, comprising:

[0069] S1. For each Tibetan syllable of the Tibetan syllable sequence to be subject to entity recognition, a pre-constructed Tibetan vocabulary dictionary is used for matching to extract a potential Tibetan vocabulary list;

[0070] S2, sequentially splicing the Tibetan syllable sequence and the potential Tibetan vocabulary list, and marking the start position and end position of each element of the spliced ​​sequence text;

[0071] The starting position and ending position numbers of each syllable element are both the position index of the syllable element in the Tibetan syllable sequence; the starting position number of each Tibetan vocabulary element is the position index of the first Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence, and the ending position number is the position index of the last Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence;

[0072] S3, based on the numbers of the starting position and the ending position of each element, calculate the relative position relationship between any two elements;

[0073] S4. Based on the relative position relationship between every two elements, obtain the corresponding relative position code.

[0074] The embodiment of the present invention at least integrates Tibetan syllables and Tibetan vocabulary information, which can improve the effect of the Tibetan named entity recognition task to a certain extent.

[0075] In an optional embodiment, if Figure 2 As shown, the embodiment of the present invention also includes:

[0076] S5. For each Tibetan syllable in the Tibetan syllable sequence to be subject to entity recognition, a pre-constructed Tibetan syllable component dictionary is used for matching to extract component features of each Tibetan syllable;

[0077] S6, performing an embedding operation on the component features of each Tibetan syllable to obtain Tibetan syllable component feature information;

[0078] S7. Fusing the relative position code and the Tibetan syllable component feature information to obtain multi-dimensional Tibetan data features integrating Tibetan syllables, Tibetan vocabulary and Tibetan syllable components.

[0079] The embodiment of the present invention effectively integrates Tibetan syllables, vocabulary and syllable component information, which can greatly improve the effect of the Tibetan named entity recognition task.

[0080] In an optional embodiment, if Figure 3 As shown, the embodiment of the present invention also includes:

[0081] S8. Using the multivariate Tibetan data features as input to a pre-trained model to perform a Tibetan named entity recognition task.

[0082] The Tibetan named entity recognition method integrating multi-feature representation provided by the embodiment of the present invention has better recognition task performance and faster reasoning speed than the existing Tibetan named entity recognition method, and has obvious effectiveness and practicality.

[0083] Next, we will introduce the steps of the above scheme in detail:

[0084] In step S1, for each Tibetan syllable in the Tibetan syllable sequence to be subject to entity recognition, a pre-constructed Tibetan vocabulary dictionary is used for matching to extract a potential Tibetan vocabulary list.

[0085] For example, the embodiment of the present invention uses Word2Vec to process 1.76GB of Tibetan text, and constructs three 50-dimensional word vector dictionaries of different granularities to capture the contextual relationship between words:

[0086] Monosyllabic dictionary: contains 22,973 Tibetan syllables. It helps to understand Tibetan and improve the language model.

[0087] Disyllabic Dictionary: It consists of 113,670 disyllabic Tibetan words. This enhances the understanding of the semantics of disyllabic words.

[0088] Comprehensive dictionary: Contains single, double and multi-syllable Tibetan words, with a total of 909,401 words. It provides strong support for complex text analysis.

[0089] During specific implementation, the above-mentioned comprehensive dictionary may be used as the Tibetan vocabulary dictionary herein.

[0090] For the Tibetan syllable sequence X{x1,x2,x3,…,x n-1 ,x n}, match all Tibetan syllable sequences with the Tibetan vocabulary dictionary and extract the potential Tibetan vocabulary list, Y{x1x2,x1x2x3,x1x2x3x4…x n-1 x n ,x2x3,x2x3x4…x n-1 x n ,…x n-1 x n}.

[0091] Wherein, x represents a Tibetan syllable, and the subscript (1, 2, 3, ..., n-1, n) represents the position index of the Tibetan syllable corresponding to the Tibetan syllable sequence.

[0092] Exemplarily, this step specifically uses the sentence to be recognized as an entity Take the translation of “Qinghai Buda Technology Company” as an example, and match the Tibetan syllable sequence with the Tibetan vocabulary dictionary.

[0093] In step S2, the Tibetan syllable sequence and the potential Tibetan vocabulary list are sequentially spliced, and each element of the spliced ​​sequence text is numbered with a start position and an end position.

[0094] This step sequentially concatenates the Tibetan syllable sequence and the potential Tibetan vocabulary list. The concatenation result can be expressed as {x1, x2, …, x n ,x1x2,x1x2x3,x1x2x3x4…x n-1 x n ,…x n-1 x n}, and mark the starting and ending positions of each element in the splicing result.

[0095] Among them, the starting position and ending position numbers of each syllable element are the position index of the syllable element in the Tibetan syllable sequence; the starting position number of each Tibetan vocabulary element is the position index of the first Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence, and the ending position number is the position index of the last Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence.

[0096] Exemplarily, still continuing the example in step S1, this step identifies a list of all potential words related to the Tibetan syllable sequence. Corresponding translation: [Qinghai, Qinghai Lake, Buda Technology, Buda Technology Company, Technology Company], the sequence text obtained by splicing Corresponding translation: [Qinghai,Buda,Science,Technology,Company,Qinghai,Qinghai Lake,Buda Technology,Buda Technology Company,Technology Company], after numbering the start and end positions of the sequence text, we can get the following: Figure 4 The flat network structure shown.

[0097] In step S3, the relative position relationship between any two elements is calculated based on the numbers of the start position and the end position of each element.

[0098] This step describes the relative position relationship between any two elements by calculating four relative distances between them:

[0099]

[0100] Among them, start[i] and end[i] represent the elements X respectively. i The starting and ending position numbers of i is the i-th element in the concatenation result;

[0101] start[j] and end[j] represent the elements X j The starting and ending position numbers of j is the jth element in the concatenation result;

[0102] Represents element X i and X j The distance from the starting position;

[0103] Represents element X i The starting position of the element X j The distance to the end position;

[0104] Represents element X i The end position of the element X j The distance from the starting position;

[0105] Represents element X i The end position of the element X j The distance to the end position.

[0106] Exemplarily, still continuing the example in step S1, this step calculates four relative distances between any two elements to describe their mutual relationship (including intersection, inclusion and separation), combined with Figure 4 Finally, the distance summary tables shown in Tables 1 to 4 are obtained.

[0107] Table 1 Summary table

[0108]

[0109]

[0110] Table 2 Summary table

[0111] 0 1 2 3 4 5 6 7 0 0 2 2 4 -1 0 1 2 3 4 5 6 -1 -1 1 1 3 -2 -1 0 1 2 3 4 5 -2 -2 0 0 2 -3 -2 -1 0 1 2 3 4 -3 -3 -1 -1 1 -4 -3 -2 -1 0 1 2 3 -4 -4 -2 -2 0 -5 -4 -3 -2 -1 0 1 2 -5 -5 -3 -3 -1 -6 -5 -4 -3 -2 -1 0 1 -6 -6 -4 -4 -2 -7 -6 -5 -4 -3 -2 -1 0 -7 -7 -5 -5 -3 -1 0 1 2 3 4 5 6 0 0 2 2 4 -2 -1 0 1 2 3 4 5 0 0 2 2 4 -5 -4 -3 -2 -1 0 1 2 -2 -2 0 0 2 -7 -6 -5 -4 -3 -2 -1 0 -2 -2 0 0 2 -7 -6 -5 -4 -3 -2 -1 0 -4 -4 -2 -2 0

[0112] Table 3 Summary table

[0113] 0 1 2 3 4 5 6 7 1 0 2 2 4 -1 0 1 2 3 4 5 6 -1 -1 1 1 3 -2 -1 0 1 2 3 4 5 -2 -2 0 0 2 -3 -2 -1 0 1 2 3 4 -3 -3 -1 -1 1 -4 -3 -2 -1 0 1 2 3 -4 -4 -2 -2 0 -5 -4 -3 -2 -1 0 1 2 -5 -5 -3 -3 -1 -6 -5 -4 -3 -2 -1 0 1 -6 -6 -4 -4 -2 -7 -6 -5 -4 -3 -2 -1 0 -7 -7 -5 -5 -3 0 1 2 3 4 5 6 7 0 0 2 2 4 0 1 2 3 4 5 6 7 0 0 2 2 4 -2 -1 0 1 2 3 4 5 -2 -2 0 0 2 -2 -1 0 1 2 3 4 5 -2 -2 0 0 2 -4 -3 -2 -1 0 1 2 3 -4 -4 -2 -2 0

[0114] Table 4 Summary table

[0115] 0 1 2 3 4 5 6 7 1 2 5 7 7 -1 0 1 2 3 4 5 6 0 1 4 6 6 -2 -1 0 1 2 3 4 5 -1 0 3 5 5 -3 -2 -1 0 1 2 3 4 -2 -1 2 4 4 -4 -3 -2 -1 0 1 2 3 -3 -2 1 3 3 -5 -4 -3 -2 -1 0 1 2 -4 -3 0 2 2 -6 -5 -4 -3 -2 -1 0 1 -5 -4 -1 1 1 -7 -6 -5 -4 -3 -2 -1 0 -6 -5 -2 0 0 0 1 2 3 4 5 6 7 0 0 2 0 0 0 1 2 3 4 5 6 7 0 0 2 2 4 -2 -1 0 1 2 3 4 5 -2 -2 0 0 2 -2 -1 0 1 2 3 4 5 -2 -2 0 0 2 -4 -3 -2 -1 0 1 2 3 -4 -4 -2 -2 0

[0116] In step S4, based on the relative position relationship between every two elements, the corresponding relative position code is obtained.

[0117] In this step, the features after the embedding operation of the four relative distances are added together to obtain the relative position code, thereby integrating the relevant vocabulary information features into the Tibetan syllables.

[0118] In step S5, for each Tibetan syllable in the Tibetan syllable sequence to be subject to entity recognition, a pre-constructed Tibetan syllable component dictionary is used for matching to extract the constituent component features of each Tibetan syllable.

[0119] It should be noted that each Tibetan syllable is composed of a base character or a base character based on the strict Tibetan grammar, which is composed of the components of the front added character, the upper added character, the lower added character, the vowel, the back added character, and the double back added character. That is, each Tibetan syllable is composed of a base character or a base character based on 1-6 components. Figure 5 As shown, Figure 2 Shown This Tibetan syllable is composed of the six components mentioned above.

[0120] Exemplarily, the embodiment of the present invention constructs a dictionary containing 18776 Tibetan syllables and their components, which covers almost all syllables that may appear in Tibetan text. Each syllable is decomposed in detail into components such as the front added character, the upper added character, the lower added character, the vowel, the back added character, and the double back added character based on the base character. The dictionary records the components of each syllable in detail, providing strong support for in-depth understanding and analysis of Tibetan syllables.

[0121] For each Tibetan syllable in the Tibetan syllable sequence to be subject to entity recognition, a Tibetan syllable component dictionary is used for matching to extract the component features of each Tibetan syllable.

[0122] Exemplarily, still continuing the example in step S1, this step is based on the sentence to be recognized as an entity Taking the translation of “Qinghai Buda Technology Company” as an example, the Tibetan syllable sequence in it is matched with the Tibetan syllable component dictionary to obtain Table 5.

[0123] Table 5 Tibetan syllables and their components

[0124]

[0125] In step S6, an embedding operation is performed on the component features of each Tibetan syllable to obtain Tibetan syllable component feature information.

[0126] This step performs the encoding operation of the Tibetan syllable component features to obtain the corresponding feature information.

[0127] In step S7, the relative position code and the Tibetan syllable component feature information are fused to obtain multi-dimensional Tibetan data features that fuse Tibetan syllables, Tibetan words, and Tibetan syllable components.

[0128] In this step, the relative position code and the Tibetan syllable component feature information are specifically spliced ​​to finally obtain multi-dimensional Tibetan data features integrating Tibetan syllables, Tibetan vocabulary and Tibetan syllable components.

[0129] In step S8, the multivariate Tibetan data features are used as input to a pre-trained model to perform a Tibetan named entity recognition task.

[0130] Exemplarily, the embodiment of the present invention selects a conditional random field model for the Tibetan named entity recognition task.

[0131] So far, the embodiment of the present invention has completed the entire process of the multi-dimensional Tibetan data feature fusion method and the Tibetan named entity recognition task.

[0132] Embodiment 2:

[0133] The embodiment of the present invention provides a multi-dimensional Tibetan data feature fusion system, including:

[0134] A vocabulary extraction module, for matching each Tibetan syllable of the Tibetan syllable sequence to be subject to entity recognition with a pre-built Tibetan vocabulary dictionary to extract a potential Tibetan vocabulary list;

[0135] A numbering module is used to sequentially splice the Tibetan syllable sequence and the potential Tibetan vocabulary list, and to mark the start position and end position of each element of the spliced ​​sequence text;

[0136] The starting position and ending position numbers of each syllable element are both the position index of the syllable element in the Tibetan syllable sequence; the starting position number of each Tibetan vocabulary element is the position index of the first Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence, and the ending position number is the position index of the last Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence;

[0137] A relationship calculation module, used to calculate the relative position relationship between any two elements based on the numbers of the starting position and the ending position of each element;

[0138] The position coding module is used to obtain the corresponding relative position coding based on the relative position relationship between every two elements.

[0139] In an optional implementation manner, the embodiment of the present invention further includes:

[0140] A component extraction module is used to match each Tibetan syllable in the Tibetan syllable sequence to be subject to entity recognition with a pre-built Tibetan syllable component dictionary to extract component features of each Tibetan syllable;

[0141] A component encoding module is used to embed the component features of each Tibetan syllable to obtain the component feature information of the Tibetan syllable;

[0142] The feature fusion module is used to fuse the relative position code and the Tibetan syllable component feature information to obtain multi-dimensional Tibetan data features that fuse Tibetan syllables, Tibetan words and Tibetan syllable components.

[0143] In an optional implementation manner, the embodiment of the present invention further includes:

[0144] The entity recognition module is used to use the multivariate Tibetan data features as input to a pre-trained model to perform a Tibetan named entity recognition task.

[0145] In an optional implementation manner, the method of calculating the relative position relationship between any two elements based on the numbers of the starting positions and ending positions of the elements in the embodiment of the present invention includes:

[0146]

[0147] Among them, start[i] and end[i] represent the elements X respectively. i The starting and ending position numbers of i is the i-th element in the concatenation result;

[0148] start[j] and end[j] represent the elements X j The starting and ending position numbers of j is the jth element in the concatenation result;

[0149] Represents element X i and X j The distance from the starting position;

[0150] Represents element X i The starting position of the element X j The distance to the end position;

[0151] Represents element X i The end position of the element X j The distance from the starting position;

[0152] Represents element X iThe end position of the element X j The distance to the end position.

[0153] Embodiment 3:

[0154] An embodiment of the present invention provides a storage medium storing a computer program for multi-dimensional Tibetan data feature fusion, wherein the computer program enables a computer to execute the multi-dimensional Tibetan data feature fusion method as described in Example 1.

[0155] Embodiment 4:

[0156] An embodiment of the present invention provides an electronic device, including:

[0157] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the multivariate Tibetan data feature fusion method as described in Example 1.

[0158] It is understandable that the multi-Tibetan data feature fusion system, storage medium and electronic device provided in the embodiments of the present invention correspond to the multi-Tibetan data feature fusion method provided in the embodiments of the present invention, and the explanations, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the multi-Tibetan data feature fusion method, which will not be repeated here.

[0159] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0160] 1. Based on Tibetan syllables, the features of Tibetan vocabulary and Tibetan syllable components are integrated, and a multi-dimensional Tibetan data feature fusion method is adopted. Compared with the method based on embedding only Tibetan syllables, it effectively solves the problem of Tibetan vocabulary information and Tibetan syllable component information being ignored, and improves the performance of Tibetan named entity recognition.

[0161] 2. A Tibetan named entity recognition method that integrates multi-feature representation has better recognition task performance and faster reasoning speed than existing Tibetan named entity recognition methods, and has obvious effectiveness and practicality.

[0162] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0163] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-dimensional Tibetan data feature fusion method, characterized in that: include: For each Tibetan syllable of the Tibetan syllable sequence to be subjected to entity recognition, a pre-built Tibetan vocabulary dictionary is used for matching to extract a potential Tibetan vocabulary list; Sequentially splicing the Tibetan syllable sequence and the potential Tibetan vocabulary list, and marking the start position and end position of each element of the spliced ​​sequence text; The starting position and ending position numbers of each syllable element are both the position index of the syllable element in the Tibetan syllable sequence; the starting position number of each Tibetan vocabulary element is the position index of the first Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence, and the ending position number is the position index of the last Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence; Based on the numbers of the starting and ending positions of each element, calculate the relative position relationship between any two elements; Based on the relative position relationship between every two elements, the corresponding relative position code is obtained.

2. The multi-dimensional Tibetan data feature fusion method according to claim 1, characterized in that: Also includes: For each Tibetan syllable in the Tibetan syllable sequence to be subject to entity recognition, a pre-constructed Tibetan syllable component dictionary is used for matching to extract component features of each Tibetan syllable; Embedding the component features of each Tibetan syllable to obtain Tibetan syllable component feature information; The relative position coding and the Tibetan syllable component feature information are integrated to obtain multi-dimensional Tibetan data features integrating Tibetan syllables, Tibetan vocabulary and Tibetan syllable components.

3. The multi-dimensional Tibetan data feature fusion method as claimed in claim 2, characterized in that: Also includes: The multivariate Tibetan data features are used as input to a pre-trained model to perform a Tibetan named entity recognition task.

4. The multi-dimensional Tibetan data feature fusion method according to claim 1, characterized in that: The relative position relationship between any two elements is calculated based on the numbers of the starting position and the ending position of each element; include: Among them, start[i] and end[i] represent the elements X respectively. i The starting and ending position numbers of i is the i-th element in the concatenation result; start[j] and end[j] represent the elements X j The starting and ending position numbers of j is the jth element in the concatenation result; Represents element X i and X j The distance from the starting position; Represents element X i The starting position of the element X j The distance to the end position; Represents element X i The end position of the element X j The distance from the starting position; Represents element X i The end position of the element X j The distance to the end position.

5. A multi-dimensional Tibetan data feature fusion system, characterized in that: include: A vocabulary extraction module, for matching each Tibetan syllable of the Tibetan syllable sequence to be subject to entity recognition with a pre-built Tibetan vocabulary dictionary to extract a potential Tibetan vocabulary list; A numbering module is used to sequentially splice the Tibetan syllable sequence and the potential Tibetan vocabulary list, and to mark the start position and end position of each element of the spliced ​​sequence text; The starting position and ending position numbers of each syllable element are both the position index of the syllable element in the Tibetan syllable sequence; the starting position number of each Tibetan vocabulary element is the position index of the first Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence, and the ending position number is the position index of the last Tibetan syllable of the Tibetan vocabulary element in the Tibetan syllable sequence; A relationship calculation module, used to calculate the relative position relationship between any two elements based on the numbers of the starting position and the ending position of each element; The position coding module is used to obtain the corresponding relative position coding based on the relative position relationship between every two elements.

6. The multi-dimensional Tibetan data feature fusion system as claimed in claim 5, characterized in that: Also includes: A component extraction module is used to match each Tibetan syllable in the Tibetan syllable sequence to be subject to entity recognition with a pre-built Tibetan syllable component dictionary to extract component features of each Tibetan syllable; A component encoding module is used to embed the component features of each Tibetan syllable to obtain the component feature information of the Tibetan syllable; The feature fusion module is used to fuse the relative position code and the Tibetan syllable component feature information to obtain multi-dimensional Tibetan data features that fuse Tibetan syllables, Tibetan words and Tibetan syllable components.

7. The multi-dimensional Tibetan data feature fusion system according to claim 6, characterized in that: Also includes: The entity recognition module is used to use the multivariate Tibetan data features as input to a pre-trained model to perform a Tibetan named entity recognition task.

8. The multi-dimensional Tibetan data feature fusion system as claimed in claim 5, characterized in that: The relative position relationship between any two elements is calculated based on the numbers of the starting position and the ending position of each element; include: Among them, start[i] and end[i] represent the elements X respectively. i The starting and ending position numbers of i is the i-th element in the concatenation result; start[j] and end[j] represent the elements X j The starting and ending position numbers of j is the jth element in the concatenation result; Represents element X i and X j The distance from the starting position; Represents element X i The starting position of the element X j The distance to the end position; Represents element X i The end position of the element X j The distance from the starting position; Represents element X i The end position of the element X j The distance to the end position.

9. A storage medium, characterized in that: The computer program for multi-dimensional Tibetan data feature fusion is stored therein, wherein the computer program enables the computer to execute the multi-dimensional Tibetan data feature fusion method as claimed in any one of claims 1 to 4.

10. An electronic device, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include programs for executing the multivariate Tibetan data feature fusion method according to any one of claims 1 to 4.

Images