Method, system and storage medium for entity semantic recognition processing in cross-language text

Abstract

The present application discloses a method, system and storage medium for processing semantic recognition of entities in cross-language texts. According to the different lengths of phonemes pronounced in different language categories for different words and the same word, different composition modes and corresponding meanings of multiple phonemes are recorded, and a sentence table is constructed for multiple words corresponding to a string of pronounced phonemes. Words of multiple language categories in the sentence table are translated into the same semantic category, so that semantic recognition can be performed according to the semantics of the words. Compared with the ordinary detection according to the previous word, the present application performs semantic detection according to the user's pause situation, which can perform semantic recognition more accurately. The semantic recognition of multiple languages ​​is performed through the sentence table, which greatly reduces the computational load of the computer, and performs semantic recognition of cross-language texts more accurately and quickly. That is, the solution of the present application can translate sentences containing different language categories into sentences of preset language categories, and the translation accuracy is high.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, system and storage medium for entity semantic recognition processing in cross-language texts. Background Art

[0002] Cross-lingual Speech-to-Text (CLS2T) is a technology that can convert speech in different languages ​​into corresponding text.

[0003] The goal of this technology is to achieve the function of speech transcription and translation between different languages.

[0004] Cross-speech text recognition technology usually includes the following steps: Speech recognition: converting speech signals into text.

[0005] This step uses speech recognition technology to analyze and match the speech signal with a model and convert it into corresponding text.

[0006] Language recognition: Determine the language of the input speech.

[0007] Since cross-speech text recognition needs to process sounds in different languages, the language of the input speech needs to be determined before performing speech recognition.

[0008] Translate: Translate the recognized text.

[0009] This step uses machine translation technology to translate the recognized text from one language into another.

[0010] However, it is often necessary to input the language category before alignment detection.

[0011] However, when a string of speech contains multiple language categories and the language category corresponding to each word is unknown, detection is more difficult.

[0012] For example, when interacting, users communicate in a mixture of Chinese and English, such as "Hello! I want to go to the hospital, how do I go?" For such a scenario with multiple languages ​​mixed in, the same phoneme can represent both Chinese and English. For example, "togo" can also be recognized as the Chinese character "tugou", making it difficult to translate the language accurately. Summary of the invention

[0013] The purpose of this application is to provide a method and system for entity semantic recognition processing in cross-language texts to solve the above-mentioned problems existing in the prior art.

[0014] In a first aspect, an embodiment of the present application provides a method for processing entity semantics recognition in a cross-language text, comprising:

[0015] Obtain a speech image to be detected and a phoneme set in a database; the phoneme set in the database includes phonemes corresponding to multiple language categories in a database and Fourier images converted from the phonemes; the speech image to be detected is a Fourier image obtained by extracting features from the speech signal to be detected through Fourier transformation;

[0016] A similar phoneme set is obtained based on the voice image to be detected and a phoneme set in a database through a triangular decision structure; the similar phoneme set includes one or more similar phonemes and similar phoneme categories corresponding to the similar phonemes; the similar phonemes represent phonemes in the database that are similar to the phonemes corresponding to the voice image to be detected;

[0017] Based on the similar phoneme set, words corresponding to the phonemes are obtained, and a sentence table is constructed based on the words corresponding to the phonemes; the sentence table includes multiple elements, each element points to a word vector; the word vector includes a word key-value pair, and the word key-value pair includes the number of word phonemes and a pointer symbol; the pointer symbol is an index symbol pointing to the word corresponding to the phoneme; the number of word phonemes is one or more; the number of word phonemes represents the number of phonemes constituting a word;

[0018] According to the corresponding words in the sentence table, the words of different language categories are translated into a unified language category to obtain a translation table;

[0019] Based on the translation table, a prediction sentence is obtained according to the position of the words in the translation table and the semantics after translation into different language categories; the prediction sentence represents multiple predicted categories including multiple languages ​​that are consistent with the context.

[0020] Optionally, obtaining a similar phoneme set based on the to-be-detected speech image and a phoneme set in a database through a triangulated decision structure includes:

[0021] The speech image to be detected is segmented with a length of one phoneme as a step length to obtain a plurality of phoneme images to be detected;

[0022] Inputting the to-be-detected phoneme image into a phoneme extraction network to extract features and obtain to-be-detected phoneme features; a plurality of to-be-detected phoneme images corresponding to a plurality of to-be-detected phoneme features are obtained;

[0023] Inputting the Fourier image in the phoneme set of the database into a phoneme extraction network to extract features and obtain stored phoneme features; multiple Fourier images in the phoneme set correspond to multiple stored phoneme features;

[0024] According to a to-be-detected phoneme feature and two stored phoneme features, differences between the features are obtained to obtain a first phoneme phase difference feature, a second phoneme phase difference feature and a third phoneme phase difference feature; the first phoneme phase difference feature represents the difference between the to-be-detected phoneme feature and a stored phoneme feature; the second phoneme phase difference feature represents the difference between the to-be-detected phoneme feature and another stored phoneme feature; the third phoneme phase difference feature represents the difference between a stored phoneme feature and another stored phoneme feature;

[0025] Based on the first phoneme difference feature, the second phoneme difference feature and the third phoneme difference feature, judging whether the features of the phonemes are similar through a phoneme discrimination structure, and if so, obtaining a first stored similar phoneme;

[0026] Based on the first phoneme difference feature, the second phoneme difference feature, the third phoneme difference feature and the first stored similar phoneme, judging whether the feature differences of the phonemes are similar through the phoneme discrimination structure;

[0027] If the feature differences of the phonemes are similar, the first stored similar phoneme is used as the similar stored phoneme feature;

[0028] The phonemes corresponding to similar stored phoneme features in the phoneme set are set as similar phonemes, and the language categories corresponding to the similar phonemes are set as similar phoneme categories.

[0029] Optionally, the triangular determination structure includes a first phoneme structure, a second phoneme structure, a third phoneme structure, a phoneme discrimination structure and a first storage structure for storing similar phonemes;

[0030] The output of the first phoneme structure and the output of the second phoneme structure are inputs of the first output of the phoneme discrimination structure;

[0031] The first output of the phoneme discrimination structure is the input of the first similar phoneme storage structure;

[0032] The output of the first phoneme structure, the output of the second phoneme structure, the output of the third phoneme structure and the output of the first storage structure storing similar phonemes constitute the second three-phase difference feature and the first three-phase difference feature;

[0033] The input of the second output of the phoneme discrimination structure is the second three-phase difference feature and the first three-phase difference feature.

[0034] Optionally, obtaining words corresponding to the phonemes based on the similar phoneme set, and constructing a sentence table based on the words corresponding to the phonemes, comprises:

[0035] Obtaining a zero matrix; wherein the rows of the zero matrix represent the arrangement of phonemes in a similar phoneme set; and the rows of the zero matrix represent language categories;

[0036] Obtaining a current similar phoneme; the current similar phoneme is a phoneme in the similar phoneme set that is pre-matched with a word;

[0037] The current similar phoneme, or the current similar phoneme and one or more preceding phonemes, form a word similar phoneme set; the preceding phoneme represents a similar phoneme that is adjacent to the current similar phoneme and pronounced earlier than the current similar phoneme; the number in the word similar phoneme set is the subscript of the current similar phoneme in the similar phoneme set plus 1; the subscripts in the similar phoneme set start from 0 and increase by 1 in sequence;

[0038] Match all the values ​​in the word-similar phoneme set, determine the word corresponding to the phoneme, and obtain the detected word and the corresponding detected word language category; n word-similar phoneme sets correspond to m detected words and m corresponding detected word language categories;

[0039] According to the row corresponding to the current similar phoneme and the column corresponding to the language category of the detected word, the number of values ​​of the detected word and the corresponding word similar phoneme set is filled into the zero matrix to obtain a sentence table.

[0040] Optionally, obtaining a predicted sentence based on the translation table according to the position of words in the translation table and the semantics after translation into different language categories includes:

[0041] Construct multiple three-word sets from three adjacent words in the translation table; the three-word set includes a first word, a second word set and a third word; the first word is adjacent to the second word set in the translation table; the second word set is adjacent to the third word in the translation table; the first word and the third word are separated by the length of one word in the second word set in the adjacent table;

[0042] The words in the second word set have the same length; the word length in the second word set is a length that can match the first word and the third word;

[0043] According to the first word and the third word, a predicted second word set is obtained; the predicted second word set includes a plurality of predicted second words and corresponding second word scores; the predicted second word represents a word that is formed with the first word and the third word; the second word score represents the number of times the corresponding phrase is used;

[0044] Obtain an intersection of the second word set and the predicted second word set to obtain a matching second word set;

[0045] Using the second word score corresponding to the value in the matching second word set as the matching second word score;

[0046] Multiple three-word sets correspond to multiple matching second word sets;

[0047] Matching the words in the plurality of matching second word sets in time order from far to near to obtain a plurality of matching statements;

[0048] Add the matching second word scores corresponding to the matching sentences to obtain a matching score; multiple matching second word scores are corresponding to multiple matching scores;

[0049] The matching second word score that is greater than the other matching second word scores among the matching second word scores is used as the detection score;

[0050] The matching sentence corresponding to the detection score is used as the predicted sentence.

[0051] Optionally, obtaining a predicted second word set according to the first word and the third word includes:

[0052] Obtain a dictionary; the dictionary is a phrase containing all the multiple language types used for detection;

[0053] According to the dictionary, a phrase score is obtained; the phrase score is a score calculated for a phrase in the dictionary according to the number of times it is used;

[0054] Finding a phrase containing the first word and only two words in the dictionary as a two-word phrase;

[0055] The first word found in the dictionary is the first word, and the third word is the third word, and there are only three words in the phrase as a three-word phrase;

[0056] The second word in a two-word phrase or a three-word phrase is used as the second word;

[0057] The word score of the phrase in which the first word is located, or the word score of the phrase containing the first word and the third word, is used as the second word score;

[0058] The second word and the second word score are added to the predicted second word set.

[0059] Optionally, judging whether the features of the phonemes are similar based on the first phoneme difference feature, the second phoneme difference feature and the third phoneme difference feature through a phoneme discrimination structure, and if they are similar, obtaining first stored similar phonemes includes:

[0060] Inputting the first phoneme difference feature and the second phoneme difference feature into a phoneme discrimination structure to obtain a second discrimination value;

[0061] Inputting the first phoneme phase difference feature and the third phoneme phase difference feature into a phoneme discrimination structure to obtain a third discrimination value;

[0062] If the second discrimination value is greater than the discrimination threshold, it is determined that the features of the phonemes are similar, and the phoneme corresponding to the second phoneme difference feature is set as the first stored similar phoneme;

[0063] If the third discrimination value is greater than the discrimination threshold, it is determined that the features of the phonemes are similar, and the phoneme corresponding to the third phoneme difference feature is set as the first stored similar phoneme;

[0064] If the second discrimination value is less than or equal to the discrimination threshold, and the third discrimination value is less than or equal to the discrimination threshold, the feature of the phoneme is determined to be dissimilar.

[0065] Optionally, judging whether the feature differences of the phonemes are similar based on the first phoneme difference feature, the second phoneme difference feature, the third phoneme difference feature and the first stored similar phoneme through a phoneme discrimination structure includes:

[0066] Obtaining another stored phoneme feature of the second phoneme phase difference feature and the third phoneme phase difference feature except the stored phoneme feature corresponding to the first stored similar phoneme as a triangular discriminant phoneme feature;

[0067] using the stored phoneme features other than the triangular discriminant phoneme features in the second phoneme phase difference feature and the third phoneme phase difference feature as phase difference discriminant phoneme features;

[0068] Subtract the triangular discriminant phoneme feature from the phase difference discriminant phoneme feature to obtain a first triangular phase difference feature;

[0069] Subtract the triangular discriminant phoneme feature from the first phoneme phase difference feature to obtain a second triangular phase difference feature;

[0070] Inputting the first triangular phase difference feature and the second triangular phase difference feature into a phoneme discrimination structure to obtain a triangular discrimination value;

[0071] If the triangular discriminant value is greater than the discriminant threshold, the feature difference of the phonemes is judged to be similar;

[0072] If the triangular discriminant value is less than or equal to the discriminant threshold, the feature difference of the phoneme is determined to be dissimilar.

[0073] In a second aspect, an embodiment of the present application provides a system for processing entity semantics recognition in a cross-language text, comprising:

[0074] Acquisition module: obtains a speech image to be detected and a phoneme set in a database; the phoneme set in the database includes phonemes corresponding to multiple language categories in the database and Fourier images converted from the phonemes; the speech image to be detected is a Fourier image after the speech signal to be detected is Fourier transformed to extract features;

[0075] A triangular determination module: a triangular determination structure is used to obtain a similar phoneme set based on the voice image to be detected and a phoneme set in a database; the similar phoneme set includes one or more similar phonemes and similar phoneme categories corresponding to the similar phonemes; the similar phonemes represent phonemes in the database that are similar to the phonemes corresponding to the voice image to be detected;

[0076] Word detection module: based on the similar phoneme set, obtain the words corresponding to the phonemes, and construct a sentence table based on the words corresponding to the phonemes; the sentence table contains multiple elements, each element points to a word vector; the word vector contains a word key-value pair, and the word key-value pair includes the number of word phonemes and a pointer symbol; the pointer symbol is an index symbol pointing to the word corresponding to the phoneme; the number of word phonemes is one or more; the number of word phonemes represents the number of phonemes constituting a word;

[0077] Translation module: According to the corresponding words in the sentence table, the words in different language categories are translated into the same language category to obtain a translation table;

[0078] Semantic recognition module: Based on the translation table, according to the position of the words in the translation table and the semantics after translation into different language categories, a prediction sentence is obtained; the prediction sentence represents multiple predicted categories containing multiple languages ​​that are consistent with the context.

[0079] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for processing entity semantic recognition in cross-language text are implemented.

[0080] Compared with the prior art, the embodiments of the present application achieve the following beneficial effects:

[0081] The embodiment of the present application also provides a method, system and storage medium for entity semantic recognition processing in cross-language text, the method comprising: obtaining a speech image to be detected and a phoneme set in a database; the phoneme set in the database comprises phonemes corresponding to multiple language categories in a database and Fourier images converted from the phonemes; the speech image to be detected is a Fourier image after the features of the speech signal to be detected are extracted through Fourier transformation; a similar phoneme set is obtained based on the speech image to be detected and the phoneme set in the database through a triangular judgment structure; the similar phoneme set comprises one or more similar phonemes and similar phoneme categories corresponding to the similar phonemes; the similar phonemes represent phonemes in the database that are similar to the phonemes corresponding to the speech image to be detected; based on the similar phoneme set , obtain the word corresponding to the phoneme, and construct a sentence table based on the word corresponding to the phoneme; the sentence table contains multiple elements, each element points to a word vector; the word vector contains a word key-value pair, and the word key-value pair includes the number of word phonemes and a pointer symbol; the pointer symbol is an index symbol pointing to the word corresponding to the phoneme; the number of word phonemes is one or more; the number of word phonemes represents the number of phonemes that constitute a word; according to the corresponding words in the sentence table, the words of different language categories are translated into a unified language category to obtain a translation table; based on the translation table, according to the position of the words in the translation table and the semantics after translation of different language categories, a predicted sentence is obtained; the predicted sentence represents a plurality of predicted categories containing multiple languages ​​that conform to the context.

[0082] In the present application, a speech image to be detected and a phoneme set in a database obtained after speech recognition are first obtained; a phoneme image to be detected is obtained, and a feature that is most similar to the phoneme to be detected in the triangular judgment structure surrounding the phoneme to be detected is found through a triangular judgment structure, and then the gap between the feature and the other two features in the triangular judgment structure surrounding the phoneme to be detected is found, so as to find the phonemes in the database that are similar to the phonemes corresponding to the speech image to be detected; through the above method, the phonemes corresponding to the phonemes to be detected in the database can be detected more accurately according to the same pronunciation in a sentence, and the parts with similar pronunciation in different language categories can be distinguished; and through the association between phonemes and words in the database, words corresponding to multiple language categories and the language corresponding to the words can be obtained. language category; according to the different lengths of the phonemes of words in different language categories, different composition methods and corresponding meanings of multiple phonemes are recorded, and a sentence table is constructed for multiple words corresponding to a string of pronounced phonemes; the words of multiple language categories in the sentence table are translated into the same semantic category, so that semantic recognition can be performed according to the semantics of the words; semantic detection is performed according to the user's pause situation according to the translation table; compared with the situation where only the previous word is found for semantic matching and the pause between two words is ignored, which may be an undetected word, semantic recognition can be performed more accurately; semantic recognition of multiple languages ​​is performed through the sentence table, which can greatly reduce the amount of computer calculation, and can achieve the technical effect of more accurate and rapid semantic recognition of cross-language texts and high translation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 It is a flow chart of a method for processing entity semantics recognition in cross-language texts provided in an embodiment of the present application.

[0084] Figure 2 It is a structural schematic diagram of a triangulation determination structure in a method for processing entity semantic recognition in a cross-language text provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] The present application is described in detail below in conjunction with the accompanying drawings. Example 1

[0086] like Figure 1 As shown, the embodiment of the present application provides a method for processing entity semantics recognition in a cross-language text, the method comprising:

[0087] S101: Obtain a speech image to be detected and a phoneme set in a database; the phoneme set in the database includes phonemes corresponding to multiple language categories in a database and Fourier images converted from the phonemes; the speech image to be detected is a Fourier image after the features of the speech signal to be detected are extracted through Fourier transformation.

[0088] Among them, the phonemes in the database phoneme set are associated with the words stored in the database.

[0089] Among them, a phoneme is a concept in phonetics, referring to the smallest distinguishable phonemic unit in speech; a word can be composed of several phonemes, and different words are distinguished by the combination and arrangement of phonemes; for example, the English word "cat" is composed of three phonemes / k / - / æ / , and the word "bat" is composed of three phonemes / b / - / æ / .

[0090] S102: Through a triangular determination structure, based on the speech image to be detected and the database phoneme set, obtain a set of similar phonemes; the set of similar phonemes includes one or more similar phonemes and corresponding similar phoneme categories; the similar phonemes represent the phonemes in the database that are similar to the phonemes corresponding to the speech image to be detected.

[0091] Among them, a matching method is used to determine whether the phonemes in the database are similar to the phonemes corresponding to the speech image to be detected.

[0092] S103: Based on the set of similar phonemes, obtain the words corresponding to the phonemes, and construct a statement table based on the words corresponding to the phonemes; the statement table contains multiple elements, and each element points to a word vector; the word vector contains word key-value pairs, and the word key-value pairs include the number of phonemes in the word and a pointer symbol; the pointer symbol is an index symbol pointing to the word corresponding to the phoneme; the number of phonemes in the word is one or more; the number of phonemes in the word represents the number of phonemes that make up a word.

[0093] Among them, the word vector contains one or more word key-value pairs.

[0094] Among them, the word corresponding to the word key-value represents a word composed of one phoneme in the speech signal to be detected, or a word composed of one phoneme in the speech signal to be detected and multiple phonemes adjacent to it and having an earlier sounding time than the phoneme; the rows of the statement table correspond to the phonemes arranged in ascending order of sounding time; the columns of the statement table correspond to language categories.

[0095] Among them, the word represents a single word, such as the English "cat" and the Chinese "猫"; a statement represents a statement composed of multiple words, such as "我喜欢猫".

[0096] Among them, the method and structure of the statement table that can accurately distinguish similar phonemes are used to obtain words with different meanings by using different phonemes multiple times.

[0097] S104: According to the words corresponding in the statement table, translate the words in different language categories into a unified language category to obtain a translation table.

[0098] Among them, detection is performed by constructing a translation dictionary; for example, a dictionary between "cat" and "猫" is constructed.

[0099] Among them, according to the words corresponding to the word list, they are translated into the values corresponding to the initial language to obtain a translated word list; the language category is unified as the initial language. In this embodiment, the initial language is set to Chinese.

[0100] S105: Based on the translation table, according to the positions of the words in the translation table and the semantics after translation of different language categories, a predicted statement is obtained.

[0101] The predicted statement represents various categories of predictions that conform to the context and include multiple languages.

[0102] By adopting the above solution, statements containing different language categories can be translated into statements of a preset language category; and the translation accuracy is high.

[0103] Optionally, the obtaining of the similar phoneme set according to the to-be-detected speech image and the phoneme set through a triangular determination structure includes:

[0104] The to-be-detected speech image is segmented with the length of one phoneme as the step size to obtain a plurality of to-be-detected phoneme images.

[0105] The to-be-detected phoneme images are input into a phoneme extraction network to extract features, and to-be-detected phoneme features are obtained; a plurality of to-be-detected phoneme images correspondingly obtain a plurality of to-be-detected phoneme features.

[0106] Among them, in this embodiment, the to-be-detected speech image is a Hidden Markov Model (HMM).

[0107] Among them, one phoneme corresponding to the to-be-detected speech image corresponds to one to-be-detected phoneme feature.

[0108] The Fourier images in the phoneme set are input into a phoneme extraction network to extract features, and stored phoneme features are obtained; a plurality of Fourier images in the phoneme set correspondingly obtain a plurality of stored phoneme features.

[0109] According to one to-be-detected phoneme feature and two stored phoneme features, the differences between the features pairwise are obtained to obtain a first phoneme difference feature, a second phoneme difference feature, and a third phoneme difference feature; the first phoneme difference feature represents the gap between the to-be-detected phoneme feature and one stored phoneme feature; the second phoneme difference feature represents the gap between the to-be-detected phoneme feature and another stored phoneme feature; the third phoneme difference feature represents the gap between one stored phoneme feature and another stored phoneme feature.

[0110] Among them, the phoneme feature a to be detected is subtracted from the stored phoneme feature b to obtain the first phoneme difference feature; the phoneme feature a to be detected is subtracted from the stored phoneme feature c to obtain the second phoneme difference feature; the stored phoneme feature b is subtracted from the stored phoneme feature c to obtain the third phoneme difference feature; a, b, and c represent codes and have no mathematical meaning; the phoneme features corresponding to a, b, and c are not the same phoneme features.

[0111] The three phoneme difference features include two features for judging whether the current phoneme to be detected is similar to the phonemes in other databases and a feature for judging the similarity between two phonemes in one database.

[0112] Based on the first phoneme difference feature, the second phoneme difference feature and the third phoneme difference feature, it is determined whether the features of the phonemes are similar, and if so, a first stored similar phoneme is obtained.

[0113] It is determined whether the feature differences of the phonemes are similar based on the first phoneme difference feature, the second phoneme difference feature and the third phoneme difference feature.

[0114] If the features of the phonemes are similar and the feature differences of the phonemes are similar, similar stored phoneme features are obtained.

[0115] The phonemes corresponding to similar stored phoneme features in the phoneme set are set as similar phonemes, and the language categories corresponding to the similar phonemes are set as similar phoneme categories.

[0116] The phoneme set represents the Fourier image converted from the phonemes corresponding to a language category in the database.

[0117] The similar phoneme category corresponds to a phoneme set, indicating a language category corresponding to the phoneme set.

[0118] Through the above method, if a phoneme to be detected is similar to a phoneme in a database, then another phoneme in the database is similar to the difference between the two; so similar features are found first, and then the same situation of the gap between the features is found.

[0119] Optionally, the triangular determination structure includes a first phoneme structure, a second phoneme structure, a third phoneme structure, a phoneme discrimination structure and a first similar phoneme storage structure.

[0120] Wherein, the triangular determination structure is as follows: Figure 2 shown.

[0121] In this embodiment, the phoneme discrimination structure has a built-in Euclidean distance function, and the Euclidean distance is calculated for the corresponding subscript value, and the obtained value is used as the difference feature, such as the subscript value corresponding to the first phoneme difference feature.

[0122] Among them, the input of the first phoneme structure is a phoneme feature to be detected, which is used to store a phoneme feature to be detected; the input of the second phoneme structure is a stored phoneme feature, which is used to store a stored phoneme feature; the input of the third phoneme structure is another stored phoneme feature, which is used to store another stored phoneme feature; the stored phoneme features in the second phoneme structure and the third phoneme structure are two stored phoneme features with different meanings.

[0123] The output of the first phoneme structure and the output of the second phoneme structure are inputs of the first output of the phoneme discrimination structure.

[0124] The first output of the phoneme discrimination structure is the input of the first similar phoneme storage structure.

[0125] The output of the first phoneme structure, the output of the second phoneme structure, the output of the third phoneme structure and the output of the first similar phoneme storage structure constitute the second three-phase difference feature and the first three-phase difference feature.

[0126] Wherein, the first stored similar phonemes are stored in the first stored similar phonemes storage structure.

[0127] The input of the second output of the phoneme discrimination structure is the second three-phase difference feature and the first three-phase difference feature.

[0128] The phoneme discrimination structure is used twice.

[0129] Optionally, the detecting words corresponding to the phonemes according to the similar phoneme set to obtain a sentence table includes:

[0130] A zero matrix is ​​obtained; the rows of the zero matrix represent the arrangement of phonemes in a similar phoneme set; and the rows of the zero matrix represent language categories.

[0131] Among them, the zero matrix is ​​a two-dimensional matrix; the number of rows of the zero matrix is ​​the number of phonemes corresponding to the fixed length of the speech image to be detected, such as 20 in this embodiment; the number of columns of the zero matrix is ​​the number of fixed language categories for detection; such as there are only two language categories, English and Chinese, in this embodiment; then the zero matrix in this embodiment is a two-dimensional matrix of 20x20.

[0132] A current similar phoneme is obtained; the current similar phoneme is a phoneme in the similar phoneme set that is pre-matched with a word.

[0133] The current similar phoneme, or the current similar phoneme and one or more preceding phonemes, are combined into a word similar phoneme set.

[0134] The front phoneme represents a phoneme that is adjacent to the current similar phoneme and is pronounced earlier than the current similar phoneme; the number in the set of word similar phonemes is the subscript of the current similar phoneme in the set of similar phonemes plus 1; the subscript in the set of similar phonemes starts from 0 and increases by 1 in sequence.

[0135] Among them, the subscript of the set starts from 0 and is set for the computer; it can be modified; the number in the set of word similar phonemes represents multiple phonemes that can form words.

[0136] Match all the values in the set of word similar phonemes, judge the words corresponding to the phonemes, and obtain the detected words and the corresponding detected word language categories; n sets of word similar phonemes correspond to obtain m detected words and m corresponding detected word language categories.

[0137] Among them, n is greater than m, and both n and m are natural numbers greater than 0; it means that the number of the set of word similar phonemes and the detected words is not equal.

[0138] According to the row corresponding to the current similar phoneme and the column corresponding to the detected word language category, fill the number of values of the detected word and the corresponding set of word similar phonemes into the zero matrix to obtain the statement table.

[0139] Among them, if the set of similar phonemes includes two phonemes "[jɛ]" and "[s]", "[jɛ]" can be "yeah [jɛ-]" in English or "叶 [[yè]]" in Chinese. Then, in the first row of the zero matrix corresponding to the English language category, fill in the label "[(1, 1)]" representing "yeah" in the first column corresponding to [jɛ], that is, it means that in the pronunciation of "[jɛ]" in English, only one index symbol "1" corresponding to "yeah" is found in this embodiment, and the number of key-value pairs in the unit vector is 1; in the second row of the zero matrix corresponding to the Chinese language category, fill in the label "[(1, 2)]" representing "叶" in the first column corresponding to the first phoneme [jɛ], that is, it means that in the pronunciation of "[jɛ]" in Chinese, only one index symbol "2" corresponding to "叶" is found in this embodiment, and the number of key-value pairs in the unit vector is 1; in the first row of the zero matrix corresponding to the English language category, fill in the label "[(2, 3)]" representing "yes" in the second column corresponding to "[jɛs]", that is, it means that in the pronunciation of "[jɛs]" in English, only one index symbol "3" corresponding to "yes" is found in this embodiment, and the number of key-value pairs in the unit vector is 1; in the second row of the zero matrix corresponding to the Chinese language category, fill in the label "[(1, 4)]" representing "私" in the second column corresponding to [s], that is, it means that in the pronunciation of "[s]" in Chinese, only one index symbol "4" corresponding to "私" is found in this embodiment, and the number of key-value pairs in the unit vector is 1.

[0140] Among them, the statement table is shown in Table 1 below:

[0141] Table 1

[0142]

[0143] Table 1 shows the representation of a statement table.

[0144] The statement table is a matrix calculation when the computer calculates.

[0145] Through the above method, the median structures of the similar phoneme set are matched in sequence to obtain multiple matching words; according to different languages, a phoneme is added each time, and then multiple corresponding technologies that can form words are found in the database; the construction of the statement table greatly reduces the amount of calculation in subsequent calculations.

[0146] Optionally, obtaining a predicted sentence based on the translation table according to the position of words in the translation table and the semantics after translation into different language categories includes:

[0147] Three adjacent words in the translation table are used to construct a three-word set; the three-word set includes a first word, a second word set and a third word; the first word is adjacent to the second word set in the translation table; the second word set is adjacent to the third word in the translation table; the first word and the third word are separated by the length of one word in the second word set in the adjacent table.

[0148] Among them, the length of the second word changes with the change of the first word and the second word, and the obtained first word, second word set and third word are different; for example, the first word corresponds to the first row of the translation table, and the third word corresponds to the third and fourth rows of the translation table; then the second row corresponding to the translation table and the second word are extracted and processed as the word with the row number 1 corresponding to the current similar phoneme to form the second word set.

[0149] The words in the second word set have the same length; the word length in the second word set is a length that can match the first word and the third word.

[0150] A predicted second word set is obtained based on the first word and the third word; the predicted second word set includes multiple predicted second words and corresponding second word scores; the predicted second word represents a word that constitutes a word with the first word and the third word; the second word score represents the number of times the corresponding phrase is used.

[0151] The intersection of the second word set and the predicted second word set is obtained to obtain a matching second word set.

[0152] The second word score corresponding to the value in the matching second word set is used as the matching second word score.

[0153] Multiple three-word sets correspond to multiple matching second word sets.

[0154] The words in the plurality of matching second word sets are matched in time order from far to near to obtain a plurality of matching statements.

[0155] Among them, the number of rows corresponding to the current similar phonemes, the values ​​corresponding to multiple matching second word sets during the matching process are adjacent instead of having other phonemes in the middle; for example, the third line in the sentence table is (1, 4), and the fourth line is (2, 5); the translated meanings corresponding to "2" and "5" can form a phrase, but the fourth line (2, 5) means that it contains the phonemes in the third line; the third line needs to be (1, 4) and the fourth line needs to be (1, 5) to be able to match.

[0156] Among them, a1 obtains a predicted second word set including b1, b2, and b3, 3 predicted second words; b1 is taken as the new first word, and a predicted second word set includes c1 and c2; b2 is taken as the new first word, and 0 predicted second words are obtained; b3 is taken as the new first word, and 0 predicted second words are obtained. Then the matching statements are "a1, b1, c1" and "a1, b1, c2".

[0157] The matching second word scores corresponding to the matching sentences are added together to obtain a matching score; multiple matching second word scores correspond to multiple matching scores.

[0158] The matching second word score that is greater than the other matching second word scores among the matching second word scores is used as the detection score.

[0159] The matching sentence corresponding to the detection score is used as the predicted sentence.

[0160] Through the above method, three words are selected according to semantic detection phrases with a step size of 1; judgments are made in sequence according to time; matching is performed according to corresponding words to obtain multiple semantic phrases; and different meanings are connected with the context to find a more accurate meaning.

[0161] Optionally, obtaining a predicted second word set according to the first word and the third word includes:

[0162] A dictionary is obtained; the dictionary is a phrase group containing all the multiple language types used for detection.

[0163] According to the dictionary, a phrase score is obtained; the phrase score is a score calculated for a phrase in the dictionary according to the number of times it is used.

[0164] Among them, the number of uses represents the number of repetitions of the phrase used by the user in the computer.

[0165] Among them, in this embodiment, the sigmod function is used for scoring.

[0166] Among them, for example, the score of "yes" is 0.11, indicating that the proportion of using "yes" or "yes" or other languages representing "yes" from the starting recording time point to the current time point in the computer is 0.11.

[0167] Find a phrase that contains the first word and only contains two words in the dictionary as a two-word phrase.

[0168] Among them, if the first word is "coconut", then a two-word phrase such as "coconut" that contains the first word and has two words can be found.

[0169] Find a phrase that has the first word as the first word, the third word as the third word, and only has three words in the dictionary as a three-word phrase.

[0170] Among them, among the multiple phrases stored in the database, find a phrase that has the first word as "coconut" and the second word as "shell" and has 3 words, such as "coconut shell" as a three-word phrase.

[0171] Take the second word in the two-word phrase or the three-word phrase as the second word.

[0172] Among them, take the "zi" corresponding to the second word of "coconut" as the second word, and take the "zi" corresponding to the second word of "coconut shell" as the second word; use the method of predicting the next word with the first word in the two-word phrase and sandwiching and predicting the second word with the first word and the third word in the three-word phrase to find the appropriate word and add it to the second word set.

[0173] Among them, one second word corresponds to one second word score and one phrase in the dictionary.

[0174] Take the word score of the phrase where the first word is located, or the word score of the phrase that contains the first word and the third word, as the second word score.

[0175] Add the second word and the second word score to the predicted second word set.

[0176] Optionally, based on the first phoneme difference feature, the second phoneme difference feature, and the third phoneme difference feature, determine whether the features of the phonemes are similar. If they are similar, obtain the first stored similar phonemes, including:

[0177] Input the first phoneme difference feature and the second phoneme difference feature into the phoneme discrimination structure to obtain a second discrimination value.

[0178] The first phoneme phase difference feature and the third phoneme phase difference feature are input into a phoneme discrimination structure to obtain a third discrimination value.

[0179] If the second discrimination value is greater than the discrimination threshold, it is determined that the features of the phonemes are similar, and the phoneme corresponding to the second phoneme difference feature is set as the first stored similar phoneme.

[0180] If the third discrimination value is greater than the discrimination threshold, it is determined that the features of the phonemes are similar, and the phoneme corresponding to the third phoneme difference feature is set as the first stored similar phoneme.

[0181] In this embodiment, the discrimination threshold is 0.9.

[0182] If the second discrimination value is less than or equal to the discrimination threshold, and the third discrimination value is less than or equal to the discrimination threshold, the feature of the phoneme is determined to be dissimilar.

[0183] Optionally, judging whether the feature differences of the phonemes are similar based on the first phoneme difference feature, the second phoneme difference feature and the third phoneme difference feature, and if so, obtaining the second stored similar phonemes includes:

[0184] Another stored phoneme feature of the second phoneme phase difference feature and the third phoneme phase difference feature except the stored phoneme feature corresponding to the first stored similar phoneme is obtained as a triangular discriminant phoneme feature.

[0185] The stored phoneme features other than the triangle determination phoneme features of the second phoneme phase difference features and the third phoneme phase difference features are used as phase difference determination phoneme features.

[0186] The phase difference determination phoneme feature is subtracted from the triangle determination phoneme feature to obtain a first triangle phase difference feature.

[0187] The first phoneme phase difference feature is subtracted from the triangular discriminant phoneme feature to obtain a second triangular phase difference feature.

[0188] The first triangular phase difference feature and the second triangular phase difference feature are input into a phoneme discrimination structure to obtain a triangular discrimination value.

[0189] If the triangular discriminant value is greater than the discriminant threshold, the feature difference of the phonemes is determined to be similar.

[0190] If the triangular discriminant value is less than or equal to the discriminant threshold, the feature difference of the phoneme is determined to be dissimilar. Example 2

[0191] The present application embodiment provides another method for processing entity semantics recognition in cross-language texts, the method comprising:

[0192] A speech image to be detected and a phoneme set in a database are obtained; the phoneme set in the database includes phonemes corresponding to multiple language categories in a database and Fourier images converted from the phonemes; the speech image to be detected is a Fourier image after the features of the speech signal to be detected are extracted through Fourier transformation.

[0193] Through the triangular decision structure, a similar phoneme set is obtained based on the voice image to be detected and the phoneme set in the database; the similar phoneme set includes one or more similar phonemes and similar phoneme categories corresponding to the similar phonemes; the similar phonemes represent phonemes in the database that are similar to the phonemes corresponding to the voice image to be detected.

[0194] Based on the similar phoneme set, words corresponding to the phonemes are obtained, and a sentence table is constructed based on the words corresponding to the phonemes; the sentence table is a pointer table, which contains multiple elements, each element pointing to a word vector; the word vector contains a phoneme in the voice signal to be detected, or a phoneme in the voice signal to be detected and multiple word pairs consisting of multiple adjacent phonemes that are pronounced earlier than the phoneme; the word pair includes a set of the number of phonemes contained in a word and a pointer symbol pointing to the word; the rows of the sentence table correspond to the phonemes arranged in order of the time of pronunciation from early to late; the columns of the sentence table represent language categories.

[0195] According to the corresponding words in the sentence table, the words in different language categories are translated into a unified language category to obtain a translation table.

[0196] Based on the translation table, a prediction sentence is obtained according to the position of the words in the translation table and the semantics after translation into different language categories; the prediction sentence represents multiple predicted categories including multiple languages ​​that are consistent with the context.

[0197] As an optional implementation, a method for processing entity semantic recognition in cross-language text includes: obtaining voice information to be detected, converting the voice information into a Fourier image, splitting phonemes based on the Fourier image, and obtaining a Fourier image (phoneme graph) of a single phoneme; matching the phoneme graph with a phoneme graph in a database to obtain a phoneme corresponding to the phoneme graph; arranging the phonemes in order into a sentence phoneme table; segmenting the phonemes in the phoneme table to obtain multiple phoneme groups, each phoneme group containing one or more phonemes; the phoneme groups contain valid phoneme groups and invalid phoneme groups; performing word matching based on each phoneme group to obtain one or more words corresponding to the phoneme group and the language category of the words; and finally translating the words according to the language category to be translated to obtain a translation text or speech of the expected language category. Example 3

[0198] Based on the above-mentioned entity semantic recognition processing method in cross-language text, an embodiment of the present application also provides an entity semantic recognition processing system in cross-language text, the system including an acquisition module, a triangulation determination module, a word detection module, a translation module and a semantic recognition module.

[0199] The acquisition module is used to obtain a speech image to be detected and multiple phoneme sets; the phoneme set represents a speech image to be detected and a database phoneme set corresponding to a language category in a database; the database phoneme set contains phonemes corresponding to multiple language categories in a database and Fourier images converted from the phonemes; the speech image to be detected is a Fourier image after the features of the speech signal to be detected are extracted through Fourier transformation.

[0200] The triangulation determination module is used to obtain a similar phoneme set based on the voice image to be detected and the phoneme set in the database through a triangulation determination structure; the similar phoneme set includes one or more similar phonemes and similar phoneme categories corresponding to the similar phonemes; the similar phonemes represent phonemes in the database that are similar to the phonemes corresponding to the voice image to be detected.

[0201] The word detection module is used to obtain the words corresponding to the phonemes based on the similar phoneme set, and to construct a sentence table based on the words corresponding to the phonemes; the sentence table contains multiple elements, each element points to a word vector; the word vector contains a word key-value pair, and the word key-value pair includes the number of word phonemes and a pointer symbol; the pointer symbol is an index symbol pointing to the word corresponding to the phoneme; the number of word phonemes is one or more; the number of word phonemes represents the number of phonemes constituting a word.

[0202] The translation module is used to translate words of different language categories into a unified language category according to corresponding words in the sentence table to obtain a translation table.

[0203] The semantic recognition module is used to obtain a predicted sentence based on the translation table according to the position of the words in the translation table and the semantics after translation into different language categories; the predicted sentence represents multiple predicted categories containing multiple languages ​​that are consistent with the context. Example 4

[0204] Based on the above-mentioned storage medium for entity semantic recognition and processing in cross-language texts, an embodiment of the present application also provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for entity semantic recognition and processing in cross-language texts are implemented.

Claims

1. A method for processing entity semantics recognition in cross-language text, characterized in that: include: Obtain a speech image to be detected and a phoneme set in a database; the phoneme set in the database includes phonemes corresponding to multiple language categories in a database and Fourier images converted from the phonemes; the speech image to be detected is a Fourier image after the speech signal to be detected is subjected to Fourier transformation to extract features; A similar phoneme set is obtained based on the voice image to be detected and the phoneme set in the database through a triangular decision structure; the similar phoneme set includes one or more similar phonemes and similar phoneme categories corresponding to the similar phonemes; The similar phonemes represent phonemes in a database that are similar to the phonemes corresponding to the voice image to be detected; Based on the similar phoneme set, words corresponding to the phonemes are obtained, and a sentence table is constructed based on the words corresponding to the phonemes; The statement table includes multiple elements, each element points to a word vector; the word vector includes a word key-value pair, and the word key-value pair includes the number of word phonemes and a pointer symbol; the pointer symbol is an index symbol pointing to the word corresponding to the phoneme; The number of word phonemes is one or more; the number of word phonemes represents the number of phonemes constituting a word; According to the corresponding words in the sentence table, the words of different language categories are translated into a unified language category to obtain a translation table; Based on the translation table, a predicted sentence is obtained according to the position of the words in the translation table and the semantics after translation into different language categories; The method of obtaining a similar phoneme set based on the to-be-detected voice image and the phoneme set in the database through the triangulated decision structure includes: The speech image to be detected is segmented with a length of one phoneme as a step length to obtain a plurality of phoneme images to be detected; Inputting the to-be-detected phoneme image into a phoneme extraction network to extract features and obtain to-be-detected phoneme features; a plurality of to-be-detected phoneme images corresponding to a plurality of to-be-detected phoneme features are obtained; Inputting the Fourier image in the phoneme set of the database into a phoneme extraction network to extract features and obtain stored phoneme features; multiple Fourier images in the phoneme set correspond to multiple stored phoneme features; According to a to-be-detected phoneme feature and two stored phoneme features, differences between the features are obtained to obtain a first phoneme phase difference feature, a second phoneme phase difference feature and a third phoneme phase difference feature; the first phoneme phase difference feature represents the difference between the to-be-detected phoneme feature and a stored phoneme feature; the second phoneme phase difference feature represents the difference between the to-be-detected phoneme feature and another stored phoneme feature; the third phoneme phase difference feature represents the difference between a stored phoneme feature and another stored phoneme feature; Based on the first phoneme difference feature, the second phoneme difference feature and the third phoneme difference feature, judging whether the features of the phonemes are similar through a phoneme discrimination structure, and if so, obtaining a first stored similar phoneme; Based on the first phoneme difference feature, the second phoneme difference feature, the third phoneme difference feature and the first stored similar phoneme, judging whether the feature differences of the phonemes are similar through the phoneme discrimination structure; If the feature differences of the phonemes are similar, the first stored similar phoneme is used as the similar stored phoneme feature; The phonemes corresponding to similar stored phoneme features in the phoneme set are set as similar phonemes, and the language categories corresponding to the similar phonemes are set as similar phoneme categories; The triangular determination structure includes a first phoneme structure, a second phoneme structure, a third phoneme structure, a phoneme discrimination structure and a first storage structure for storing similar phonemes; The output of the first phoneme structure and the output of the second phoneme structure are inputs of the first output of the phoneme discrimination structure; The first output of the phoneme discrimination structure is the input of the first similar phoneme storage structure; The output of the first phoneme structure, the output of the second phoneme structure, the output of the third phoneme structure and the output of the first storage structure storing similar phonemes constitute the second three-phase difference feature and the first three-phase difference feature; The input of the second output of the phoneme discrimination structure is the second three-phase difference feature and the first three-phase difference feature.

2. The method for processing entity semantics recognition in cross-language text according to claim 1, characterized in that: The method of obtaining words corresponding to the phonemes based on the similar phoneme set and constructing a sentence table based on the words corresponding to the phonemes includes: Obtaining a zero matrix; wherein the rows of the zero matrix represent the arrangement of phonemes in a similar phoneme set; and the rows of the zero matrix represent language categories; Obtaining a current similar phoneme; the current similar phoneme is a phoneme in the similar phoneme set that is pre-matched with a word; The current similar phoneme, or the current similar phoneme and one or more preceding phonemes, form a word similar phoneme set; the preceding phoneme represents a similar phoneme that is adjacent to the current similar phoneme and pronounced earlier than the current similar phoneme; the number in the word similar phoneme set is the subscript of the current similar phoneme in the similar phoneme set plus 1; the subscripts in the similar phoneme set start from 0 and increase by 1 in sequence; Match all values ​​in the word-similar phoneme set, determine the word corresponding to the phoneme, and obtain a detected word and a corresponding detected word language category; n word-similar phoneme sets correspond to m detected words and m corresponding detected word language categories; According to the row corresponding to the current similar phoneme and the column corresponding to the language category of the detected word, the number of values ​​of the detected word and the corresponding word similar phoneme set is filled into the zero matrix to obtain a sentence table.

3. The method for processing entity semantics recognition in cross-language text according to claim 1, characterized in that: The method of obtaining a predicted sentence based on the translation table according to the position of the words in the translation table and the semantics after translation of different language categories includes: Construct multiple three-word sets from three adjacent words in the translation table; the three-word set includes a first word, a second word set and a third word; the first word is adjacent to the second word set in the translation table; the second word set is adjacent to the third word in the translation table; the first word and the third word are separated by the length of one word in the second word set in the adjacent table; The words in the second word set have the same length; the word length in the second word set is a length that can match the first word and the third word; According to the first word and the third word, a predicted second word set is obtained; the predicted second word set includes a plurality of predicted second words and corresponding second word scores; the predicted second word represents a word that constitutes a word with the first word and the third word; the second word score represents the number of times the corresponding phrase is used; Obtain an intersection of the second word set and the predicted second word set to obtain a matching second word set; Using the second word score corresponding to the value in the matching second word set as the matching second word score; Multiple three-word sets correspond to multiple matching second word sets; Matching the words in the plurality of matching second word sets in time order from far to near to obtain a plurality of matching statements; Add the matching second word scores corresponding to the matching sentences to obtain a matching score; multiple matching second word scores are corresponding to multiple matching scores; The matching second word score that is greater than the other matching second word scores among the matching second word scores is used as the detection score; and the matching sentence corresponding to the detection score is used as the predicted sentence.

4. The method for processing entity semantics recognition in cross-language text according to claim 3, characterized in that: The step of obtaining a predicted second word set according to the first word and the third word includes: Obtain a dictionary; the dictionary is a phrase containing all the multiple language types used for detection; According to the dictionary, a phrase score is obtained; the phrase score is a score calculated for a phrase in the dictionary according to the number of times it is used; Finding a phrase containing the first word and only two words in the dictionary as a two-word phrase; The first word found in the dictionary is the first word, and the third word is the third word, and there are only three words in the phrase as a three-word phrase; The second word in a two-word phrase or a three-word phrase is used as the second word; The word score of the phrase in which the first word is located, or the word score of the phrase containing the first word and the third word, is used as the second word score; The second word and the second word score are added to the predicted second word set.

5. The method for processing entity semantics recognition in cross-language text according to claim 1, characterized in that: The determining whether the features of the phonemes are similar based on the first phoneme difference feature, the second phoneme difference feature and the third phoneme difference feature through a phoneme discrimination structure, and if they are similar, obtaining first stored similar phonemes, comprises: Inputting the first phoneme difference feature and the second phoneme difference feature into a phoneme discrimination structure to obtain a second discrimination value; Inputting the first phoneme phase difference feature and the third phoneme phase difference feature into a phoneme discrimination structure to obtain a third discrimination value; If the second discrimination value is greater than the discrimination threshold, it is determined that the features of the phonemes are similar, and the phoneme corresponding to the second phoneme difference feature is set as the first stored similar phoneme; If the third discrimination value is greater than the discrimination threshold, it is determined that the features of the phonemes are similar, and the phoneme corresponding to the third phoneme difference feature is set as the first stored similar phoneme; If the second discrimination value is less than or equal to the discrimination threshold, and the third discrimination value is less than or equal to the discrimination threshold, the feature of the phoneme is determined to be dissimilar.

6. The method for processing entity semantics recognition in cross-language text according to claim 1, characterized in that: The determining whether the feature differences of the phonemes are similar based on the first phoneme difference feature, the second phoneme difference feature, the third phoneme difference feature and the first stored similar phoneme through a phoneme discrimination structure includes: Obtaining another stored phoneme feature of the second phoneme phase difference feature and the third phoneme phase difference feature except the stored phoneme feature corresponding to the first stored similar phoneme as a triangular discriminant phoneme feature; using the stored phoneme features other than the triangular discriminant phoneme features in the second phoneme phase difference feature and the third phoneme phase difference feature as phase difference discriminant phoneme features; Subtract the triangular discriminant phoneme feature from the phase difference discriminant phoneme feature to obtain a first triangular phase difference feature; Subtract the triangular discriminant phoneme feature from the first phoneme phase difference feature to obtain a second triangular phase difference feature; Inputting the first triangular phase difference feature and the second triangular phase difference feature into a phoneme discrimination structure to obtain a triangular discrimination value; If the triangular discriminant value is greater than the discriminant threshold, the feature difference of the phonemes is judged to be similar; If the triangular discriminant value is less than or equal to the discriminant threshold, the feature difference of the phoneme is determined to be dissimilar.

7. A system for identifying and processing entity semantics in cross-language texts, characterized in that: include: Acquisition module: obtains a speech image to be detected and a phoneme set in a database; the phoneme set in the database includes phonemes corresponding to multiple language categories in the database and Fourier images converted from the phonemes; the speech image to be detected is a Fourier image after the speech signal to be detected is Fourier transformed to extract features; A triangulation determination module: obtaining a similar phoneme set based on the voice image to be detected and the phoneme set in the database through a triangulation determination structure; the similar phoneme set includes one or more similar phonemes and similar phoneme categories corresponding to the similar phonemes; The similar phonemes represent phonemes in a database that are similar to the phonemes corresponding to the voice image to be detected; Word detection module: based on the similar phoneme set, obtain the words corresponding to the phonemes, and construct a sentence table based on the words corresponding to the phonemes; The statement table includes multiple elements, each element points to a word vector; the word vector includes a word key-value pair, and the word key-value pair includes the number of word phonemes and a pointer symbol; the pointer symbol is an index symbol pointing to the word corresponding to the phoneme; The number of word phonemes is one or more; the number of word phonemes represents the number of phonemes constituting a word; Translation module: According to the corresponding words in the sentence table, the words in different language categories are translated into the same language category to obtain a translation table; Semantic recognition module: based on the translation table, obtains the predicted sentence according to the position of the words in the translation table and the semantics after translation of different language categories; The method of obtaining a similar phoneme set based on the to-be-detected voice image and the phoneme set in the database through the triangulated decision structure includes: The speech image to be detected is segmented with a length of one phoneme as a step length to obtain a plurality of phoneme images to be detected; Inputting the to-be-detected phoneme image into a phoneme extraction network to extract features and obtain to-be-detected phoneme features; a plurality of to-be-detected phoneme images corresponding to a plurality of to-be-detected phoneme features are obtained; Inputting the Fourier image in the phoneme set of the database into a phoneme extraction network to extract features and obtain stored phoneme features; multiple Fourier images in the phoneme set correspond to multiple stored phoneme features; According to a to-be-detected phoneme feature and two stored phoneme features, differences between the features are obtained to obtain a first phoneme phase difference feature, a second phoneme phase difference feature and a third phoneme phase difference feature; the first phoneme phase difference feature represents the difference between the to-be-detected phoneme feature and a stored phoneme feature; the second phoneme phase difference feature represents the difference between the to-be-detected phoneme feature and another stored phoneme feature; the third phoneme phase difference feature represents the difference between a stored phoneme feature and another stored phoneme feature; Based on the first phoneme difference feature, the second phoneme difference feature and the third phoneme difference feature, judging whether the features of the phonemes are similar through a phoneme discrimination structure, and if so, obtaining a first stored similar phoneme; Based on the first phoneme difference feature, the second phoneme difference feature, the third phoneme difference feature and the first stored similar phoneme, judging whether the feature differences of the phonemes are similar through the phoneme discrimination structure; If the feature differences of the phonemes are similar, the first stored similar phoneme is used as the similar stored phoneme feature; The phonemes corresponding to similar stored phoneme features in the phoneme set are set as similar phonemes, and the language categories corresponding to the similar phonemes are set as similar phoneme categories; The triangular determination structure includes a first phoneme structure, a second phoneme structure, a third phoneme structure, a phoneme determination structure and a first storage structure for storing similar phonemes; The output of the first phoneme structure and the output of the second phoneme structure are inputs of the first output of the phoneme discrimination structure; The first output of the phoneme discrimination structure is the input of the first similar phoneme storage structure; The output of the first phoneme structure, the output of the second phoneme structure, the output of the third phoneme structure and the output of the first storage structure storing similar phonemes constitute the second three-phase difference feature and the first three-phase difference feature; The input of the second output of the phoneme discrimination structure is the second three-phase difference feature and the first three-phase difference feature.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for processing entity semantics recognition in cross-language texts according to any one of claims 1 to 6 are implemented.

Images