A method and system for mapping sign language out-of-vocabulary sign items

Abstract

The application discloses a sign language unregistered sign language word mapping method and system, comprising the following steps: constructing index information of registered sign language words based on an HNSW algorithm; constructing a mapping table of high-frequency unregistered sign language words to the registered sign language words based on the index information of the registered sign language words; establishing a matching mechanism of special style unregistered sign language words to the registered sign language words based on rules; and realizing mapping of the unregistered sign language words to be tested based on the mapping table, the matching mechanism and the index information. The method and the system can realize mapping of unregistered words to the registered sign language words, so that sign language word sequences can be understood by hearing-impaired people as much as possible.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer artificial intelligence, and particularly relates to a sign language out-of-vocabulary sign language word mapping method and system. BACKGROUND

[0002] As the language for communication of the hearing-impaired, sign language is an indispensable way of communication for them. In recent years, with the development of natural language processing technology, academic staff can fine-tune a Chinese text to sign language word sequence translation model with a pre-trained multilingual model and relatively scarce Chinese text-sign language word sequence parallel corpus. However, due to the inconsistency between the word table used to train the multilingual model and the words used in the actual communication of the deaf and mute, part of the words translated by the model may not conform to the words used in the actual communication of the hearing-impaired.

[0003] Patent document CN114840670A discloses a Chinese text to sign language word sequence translation method and device, comprising: obtaining a Chinese text to be translated, preprocessing the Chinese text to obtain a preprocessed text, translating the preprocessed text through a target translation model to obtain a corresponding sign language word sequence, and post-processing the sign language word sequence to obtain a target sign language word sequence. Although the technical solution obtains the annotated sign language word sequence corresponding to the Chinese text through a unified sign language annotation method, corrects the standard sign language word sequence, and obtains a bilingual corpus, so that the translation result of the bilingual corpus conforms to the sign language grammar, thereby improving the accuracy of the translation result in the bilingual corpus. However, for out-of-vocabulary words, the translation method still has the problem of inaccurate translation.

[0004] Patent document CN114758411A discloses a sign language translation system based on a codec, comprising: a feature extraction module for extracting features of a sign language video, the input of which is a sign language video and the output of which is features that can be decoded into a sign language word sequence; a translation module for establishing a translation model between sign language and natural language, the input of which is a sign language word sequence and the output of which is a probability vector sequence that can be decoded into a natural language word sequence. The system realizes translation of sign language words to natural language based on the translation model. Similarly, for out-of-vocabulary words, the translation method still has the problem of inaccurate translation. SUMMARY

[0005] In view of the above, the purpose of the present application is to provide a sign language out-of-vocabulary sign language word mapping method and system, which can realize mapping of out-of-vocabulary words to recorded sign language words, so that the sign language word sequence is understood by the hearing-impaired as much as possible.

[0006] To achieve the above-mentioned purpose of the application, the sign language out-of-vocabulary sign language word mapping method provided by the embodiment comprises the following steps:

[0007] constructing index information of the recorded sign words based on the HNSW algorithm;

[0008] constructing a mapping table of the high-frequency out-of-vocabulary sign words to the recorded sign words based on the index information of the recorded sign words;

[0009] establishing a matching mechanism of the special style out-of-vocabulary sign words to the recorded sign words based on rules;

[0010] implementing mapping of the to-be-tested out-of-vocabulary sign words based on the mapping table, the matching mechanism and the index information.

[0011] Preferably, the constructing index information of the recorded sign words based on the HNSW algorithm comprises:

[0012] obtaining a plurality of word vectors of each recorded sign word by using a plurality of word vector models; constructing word vector index information based on the word vectors generated by each word vector model and the HNSW algorithm, thereby generating a plurality of word vector index information; respectively using each word vector index information to find the word vector of the recorded sign word that is most similar in semantics to the word vector of the out-of-vocabulary word; and manually judging the effect of each word vector index information in taking the most similar recorded sign word in semantics and giving a corresponding weight.

[0013] Preferably, the constructing a mapping table of the high-frequency out-of-vocabulary sign words to the recorded sign words based on the index information of the recorded sign words comprises:

[0014] finding, according to each word vector index information, a plurality of word vectors of the recorded sign words that are most similar in semantics to the word vector of the high-frequency out-of-vocabulary sign word, and respectively obtaining the sign words corresponding to the word vectors of each recorded sign word;

[0015] weighting and scoring the corresponding plurality of recorded sign words according to the weight of each word vector index information, arranging the plurality of recorded sign words from high to low according to the scores, manually judging the one of the plurality of sign words that is most similar in semantics to the out-of-vocabulary sign word, and thereby constructing a mapping table of the high-frequency out-of-vocabulary words to the recorded sign words.

[0016] Preferably, the establishing a matching mechanism of the special style out-of-vocabulary sign words to the recorded sign words based on rules comprises:

[0017] when the special style out-of-vocabulary sign word is a word composed of numbers and units, converting the numbers into a combination of recorded number words and retaining the units;

[0018] when the special style out-of-vocabulary sign word is a word composed of numbers and units, converting the numbers into a combination of recorded number words and retaining the units;

[0019] When the special style unregistered sign language word is in the form of a combination of numbers and English or in pure English, the numbers and English or English are outputted in sequence according to the character granularity.

[0020] Preferably, the mapping of the to-be-tested unregistered sign language word is realized based on the mapping table, the matching mechanism and the index information, and comprises:

[0021] (a) judging whether the to-be-tested unregistered sign language word is hit by the mapping table according to the mapping table, i.e. whether the to-be-tested unregistered sign language word can be obtained according to the mapping table, if yes, outputting the hit registered sign language word, if no, executing step (b);

[0022] (b) judging whether the format of the to-be-tested unregistered sign language word is hit by the special style according to the matching mechanism, i.e. judging whether the to-be-tested unregistered sign language word is a special style unregistered sign language word, if yes, outputting the mapping result corresponding to the special style unregistered sign language word according to the matching mechanism, if no, executing step (c);

[0023] (c) retrieving a plurality of registered sign language words for the to-be-tested unregistered sign language word according to each word vector index information, and calculating the semantic similarity between the to-be-tested unregistered sign language word and the registered sign language words, if the maximum semantic similarity is higher than a threshold, outputting the registered sign language word corresponding to the maximum semantic similarity, if no, executing step (d);

[0024] (d) extracting the initial letters of the pinyin English of each character in the to-be-tested unregistered sign language word and outputting.

[0025] Preferably, the index information comprises a plurality of word vector index information, and the retrieving of the plurality of registered sign language words for the to-be-tested unregistered sign language word according to each word vector index information and the calculating of the semantic similarity between the to-be-tested unregistered sign language word and the registered sign language words comprise:

[0026] calculating the semantic similarity between the word vector of the to-be-tested unregistered sign language word and the word vector of the registered sign language word according to each word vector index information, and extracting the registered sign language words corresponding to the plurality of word vectors with the most similar semantics generated by each index information respectively;

[0027] weighting the semantic similarity of the corresponding plurality of registered sign language words according to the weight of each word vector index information to obtain the weighted semantic similarity.

[0028] Preferably, the semantic similarity is determined according to the Euclidean distance between the word vector of the unregistered sign language word and the word vector of the registered sign language word.

[0029] To achieve the above-mentioned purposes, the embodiment further provides a sign language unregistered sign language word mapping system, comprising:

[0030] an index information construction module configured to construct index information of the recorded sign language words based on the HNSW algorithm;

[0031] a mapping table construction module configured to construct a mapping table of the high-frequency out-of-vocabulary sign language words to the recorded sign language words based on the index information of the recorded sign language words;

[0032] a matching mechanism construction module configured to establish a matching mechanism of the special style out-of-vocabulary sign language words to the recorded sign language words based on rules;

[0033] a mapping module configured to realize mapping of the out-of-vocabulary sign language words based on the mapping table, the matching mechanism and the index information.

[0034] To achieve the above-mentioned purposes, the embodiment further provides a computing device, including a memory, a processor and a computer program stored in the memory and executable on the processor, the memory has a jewelry with a label, and the processor implements the steps of the above-mentioned sign language out-of-vocabulary sign language word mapping method when executing the computer program.

[0035] To achieve the above-mentioned purposes, the embodiment provides a computer readable storage medium, which stores a computer program, and the computer program is executable on the processor to implement the steps of the above-mentioned sign language out-of-vocabulary sign language word mapping method.

[0036] Compared with the prior art, the present application has at least the following beneficial effects:

[0037] The sign language out-of-vocabulary sign language word mapping method and system provided by the present application construct index information of recorded sign language words based on the HNSW algorithm, construct a mapping table of high-frequency out-of-vocabulary sign language words to recorded sign language words based on the index information of recorded sign language words, establish a matching mechanism of special style out-of-vocabulary sign language words to recorded sign language words based on rules, and form a three-layer funnel mapping system with the constructed index information, mapping table and matching mechanism. Based on the mapping system and the mapping process, accurate mapping of any out-of-vocabulary sign language word can be realized, so that as many sign language word sequences as possible can be understood by the hearing-impaired. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0039] Figure 1 is a flowchart of the sign language out-of-vocabulary sign language word mapping method provided by the embodiment;

[0040] Figure 2 This is a flowchart of the process of constructing the mapping table provided in the embodiment;

[0041] Figure 3 This is a flowchart of an example of the method for mapping unregistered sign language entries provided in the embodiments;

[0042] Figure 4 This is a schematic diagram of the structure of the unregistered sign language word mapping system provided in the embodiment;

[0043] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0045] Figure 1 This is a flowchart of the method for mapping unregistered sign language entries provided in the embodiment. For example... Figure 1 As shown, the method for mapping unregistered sign language terms provided in this embodiment of the invention includes the following steps:

[0046] S110, constructs an index of sign language entries based on the HNSW algorithm.

[0047] In this embodiment, "registered sign language entries" refers to sign language entries that have been recorded by the motion capture device; conversely, "unregistered sign language entries" refers to those that have not been registered. HNSW, short for Hierarchical Navigable Small Worlds, is an approximate K-nearest neighbor search based on a controllable hierarchical navigable small world graph. Its principle is to divide the original data vector into different spaces through mapping, and then, for large-scale search tasks, perform query operations in the space of similar vectors through a mapping function.

[0048] In this embodiment, an index of recorded sign language words is constructed based on the HNSW algorithm, including:

[0049] A plurality of word vector models are used to obtain a plurality of word vectors of each entered sign language word object; word vector index information is constructed based on the word vectors generated by each word vector model and the HNSW algorithm, thereby generating a plurality of word vector index information; each word vector index information is used to find the word vector of the entered sign language word object that is most similar in semantics to the word vector of the unregistered word; wherein, for each word vector index information, the index effect is artificially quantitatively judged when the most similar entered sign language word object is taken, and the index with higher accuracy in extracting similar word objects is given higher weight, and vice versa.

[0050] In the embodiment, the word vector model is a shallow neural network that maps words to a specific dimensional vector. The product of the model is a set of word vectors, where the closer the vectors are in space, the closer the corresponding words are in semantics and meaning. The semantic similarity is determined by calculating the Euclidean distance between the word vector of the unregistered word and the word vector of the entered sign language word object.

[0051] S120, constructing a mapping table of high-frequency unregistered sign language words to entered sign language words based on the index information of the entered sign language words.

[0052] In the embodiment, as shown in Figure 2 , constructing a mapping table of high-frequency unregistered sign language words to entered sign language words based on the index information of the entered sign language words includes:

[0053] S210, finding the word vectors of a plurality of entered sign language words that are most similar in semantics to the word vector of the high-frequency unregistered sign language word according to each word vector index information, and obtaining the corresponding sign language word object of each entered sign language word object; S220, weighting and scoring the corresponding plurality of entered sign language words according to the weight of each word vector index information; S230, arranging the plurality of entered sign language words from high to low according to the score, and manually judging the one that is most similar in semantics to the unregistered sign language word from the plurality of sign language words, thereby constructing a mapping table of high-frequency unregistered words to entered sign language words.

[0054] S130, establishing a matching mechanism of special style unregistered sign language words to entered sign language words based on rules.

[0055] In the embodiment, establishing a matching mechanism of special style unregistered sign language words to entered sign language words based on rules includes:

[0056] When the special style of the unknown sign language word is a word formed by a combination of a number and a unit, the number is transcribed into the combination of the recorded number words, and the unit is retained; when the special style of the unknown sign language word is a word formed by a combination of a number and English or a pure English word, the number and English or English are outputted in sequence according to the character granularity.

[0057] In S140, mapping of the to-be-tested unknown sign language word is realized based on the mapping table, the matching mechanism and the index information.

[0058] In the embodiment, the mapping of the to-be-tested unknown sign language word is realized based on the mapping table, the matching mechanism and the index information, which includes:

[0059] (a) determining whether the to-be-tested unknown sign language word is hit by the mapping table, i.e., whether the to-be-tested unknown sign language word can be queried to obtain the recorded sign language word corresponding to the to-be-tested unknown sign language word according to the mapping table, if yes, outputting the hit recorded sign language word, and if no, executing step (b);

[0060] (b) determining whether the format of the to-be-tested unknown sign language word is hit by the special style, i.e., determining whether the to-be-tested unknown sign language word is a special style unknown sign language word, if yes, outputting the mapping result corresponding to the special style unknown sign language word according to the matching mechanism, and if no, executing step (c);

[0061] (c) retrieving a plurality of recorded sign language words for the to-be-tested unknown sign language word according to the index information, and calculating the semantic similarity between the to-be-tested unknown sign language word and the recorded sign language words, if the maximum semantic similarity is higher than a threshold, outputting the recorded sign language word corresponding to the maximum semantic similarity, and if no, executing step (d);

[0062] Wherein, the retrieving a plurality of recorded sign language words for the to-be-tested unknown sign language word according to the index information, and calculating the semantic similarity between the to-be-tested unknown sign language word and the recorded sign language words, includes: calculating the semantic similarity between the word vector of the to-be-tested unknown sign language word and the word vector of the recorded sign language word according to each word vector index information, and extracting the word vector of the plurality of recorded sign language words most similar in semantics for each word vector index information; weighting the semantic similarity of the plurality of recorded sign language words corresponding to each word vector index information according to the weight of each word vector index information to obtain the weighted semantic similarity.

[0063] (d) extracting the English initial letter of each character in the to-be-tested unknown sign language word and outputting.

[0064] Based on the sign language unknown sign language word mapping method provided in the above embodiment, the embodiment further provides a method example, as shown in Figure 3 which includes the following steps:

[0065] Step 1, input the national common sign language dictionary and additional vocabulary entries {w Vi Using word vector model M i Generate word vectors {v Vi}

[0066] Step 2, based on the word vector {v Vi The HNSW algorithm is used to construct word vector index information. i .

[0067] Step 3: Collect high-frequency out-of-vocabulary words from different corpora. OOVi Using word vector model M i Generate word vectors for unregistered words {v OOVi}

[0068] Step 4: Repeat steps 1 and 2 to generate three word vector index information I1, I2, and I3 using three different pre-trained word vector models M1, M2, and M3.

[0069] Step 5: Based on the word vector index information I1, I2, and I3 generated in Step 4, search for the out-of-vocabulary word {v}. OOVi} From the 10 words that are most semantically similar in the already entered entries, we get {w I1_1 ,…,w I1_10},{w I2_1 ,…,w I2_10},{w I3_1 ,…,w I3_10 Semantic similarity is determined using the Euclidean distance between two vectors; then, based on index weights, the 10 most semantically similar entries are extracted from the existing vocabulary; these 10 entries are then used as a reference, and unregistered entries are manually mapped. OOVi The word is the one most semantically similar to the one in the dictionary, and a mapping table is formed from high-frequency out-of-vocabulary words to the entries.

[0070] Step 6, establish a matching mechanism based on specific styles: First, determine the non-registered words {w OOVi Is the entry a purely numerical term? If so, transcribe the numbers into the already entered numerical term combinations. Then, determine the unregistered words {w}. OOVi Is it a term that combines numbers and units? If so, transcribe the numbers into the already entered number term combination and retain the units.

[0071] Finally, determine the unregistered words {w OOVi Is it a combination of numbers and English letters? If so, output the numbers and English letters in order of character granularity.

[0072] Step 7: Input the out-of-vocabulary words generated by the translation model.

[0073] Step 8, determine whether the input OOV word of step 7 is hit by the OOV-to-PIW mapping table of step 5; if yes, output the corresponding PIW; otherwise, enter the funnel of specific text style matching.

[0074] Step 9, determine whether the input OOV word of step 7 is hit by the specific text style matching mechanism of step 6; if yes, output the corresponding result in step 6; otherwise, enter the funnel of semantic similarity matching based on word vector index.

[0075] Step 10, based on the word vector indexes I1, I2, I3 generated in step 4, find the 10 most semantically similar words of the input OOV word in the PIW, respectively, to obtain {w I1_1 ,…,w I1_10}, {w I2_1 ,…,w I2_10}, {w I3_1 ,…,w I3_10}; the semantic similarity uses the Euclidean distance between two vectors; then, based on the index weight, extract the most semantically similar PIW. If the similarity is high enough, output the corresponding PIW; otherwise, extract the English initials of each character in the OOV word.

[0076] Based on the same inventive concept, the embodiment also provides a sign language OOV sign language word mapping system 400, as shown in Figure 4 , comprising an index information construction module 410, a mapping table construction module 420, a matching mechanism construction module 430, and a mapping module 440.

[0077] The index information construction module 410 is configured to construct index information of the PIW based on the HNSW algorithm; the mapping table construction module 420 is configured to construct a mapping table of high-frequency OOV sign language words to PIW based on the index information of the PIW; the matching mechanism construction module 430 is configured to establish a matching mechanism of special style OOV sign language words to PIW based on rules; and the mapping module 440 is configured to realize mapping of OOV sign language words based on the mapping table, the matching mechanism, and the index information.

[0078] It should be noted that the sign language unregistered sign language word mapping device provided in the above embodiment should be illustrated by the above division of each functional module when performing sign language unregistered sign language word mapping, and the above functions can be completed by different functional modules according to needs, that is, the internal structure of the terminal or the server is divided into different functional modules to complete all or part of the functions described above. In addition, the sign language unregistered sign language word mapping device and the sign language unregistered sign language word mapping method provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the sign language unregistered sign language word mapping method embodiment, which will not be repeated here.

[0079] Based on the same inventive concept, the embodiment also provides a computing device, as shown in the figure, which includes a processor, an internal bus, a network interface, a memory and a storage, and of course can also include other hardware required by the business. The processor reads the corresponding computer program from the storage into the memory and then runs to implement the above-mentioned sign language unregistered sign language word mapping method, including the following steps: Figure 5

[0080] S110, constructing index information of the registered sign language words based on the HNSW algorithm;

[0081] S120, constructing a mapping table of high-frequency unregistered sign language words to registered sign language words based on the index information of the registered sign language words;

[0082] S130, establishing a matching mechanism of special style unregistered sign language words to registered sign language words based on rules;

[0083] S140, realizing the mapping of the to-be-tested unregistered sign language words based on the mapping table, the matching mechanism and the index information.

[0084] The storage can be a proximal volatile memory such as RAM, and can also be a non-volatile memory such as ROM, FLASH, floppy disk, mechanical hard disk, etc., and can also be a remote storage cloud. The processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), that is, the steps of the sign language unregistered sign language word mapping method can be realized by these processors.

[0085] Based on the same inventive concept, the embodiment also provides a computer readable storage medium having a computer program stored thereon, which computer program is executed by a processor to implement the above-mentioned sign language unregistered sign language word mapping method, including the following steps:

[0086] S110, constructing index information of the registered sign language words based on the HNSW algorithm;

[0087] ​S120, constructing a mapping table of high-frequency unregistered sign language words to registered sign language words based on the index information of the registered sign language words;

[0088] S130, establishing a matching mechanism of special style unregistered sign language words to registered sign language words based on rules;

[0089] S140, realizing mapping of the to-be-tested unregistered sign language words based on the mapping table, the matching mechanism and the index information.

[0090] The computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0091] The above detailed description of the specific embodiments has described the technical solutions and beneficial effects of the present application. It should be understood that the above description is only the most preferred embodiment of the present application and is not intended to limit the present application. Any modification, supplement and equivalent replacement within the principle range of the present application should be included in the protection scope of the present application.

Claims

1. A method for mapping unregistered sign language entries, characterized in that, The method comprises the following steps: Index information of the recorded sign language words is constructed based on an HNSW algorithm; A mapping table of high-frequency out-of-vocabulary sign language words to recorded sign language words is constructed based on the index information of the recorded sign language words, specifically comprising: finding, according to each word vector index information, multiple word vectors of the recorded sign language words that are most similar in semantics to the word vector of the high-frequency out-of-vocabulary sign language word, and respectively obtaining the sign language words corresponding to the word vectors of each recorded sign language word; weighting and scoring the multiple recorded sign language words corresponding to each word vector index information according to the weight of each word vector index information, arranging the multiple recorded sign language words from high to low according to the scores, and manually judging the one most similar in semantics to the out-of-vocabulary sign language word among the multiple sign language words, thereby constructing the mapping table of the high-frequency out-of-vocabulary words to the recorded sign language words; A matching mechanism of special style out-of-vocabulary sign language words to recorded sign language words is established based on rules, specifically comprising: when the special style out-of-vocabulary sign language word is a word composed of pure numbers, the numbers are transcribed into a combination of recorded number words; when the special style out-of-vocabulary sign language word is a word composed of numbers and units, the numbers are transcribed into a combination of recorded number words, and the units are retained; when the special style out-of-vocabulary sign language word is a word composed of numbers and English or pure English, the numbers and English or English are outputted in turn according to the character granularity; Mapping of the to-be-tested out-of-vocabulary sign language word is realized based on the mapping table, the matching mechanism and the index information, comprising: (a) judging whether the to-be-tested out-of-vocabulary sign language word is hit by the mapping table, i.e. whether the to-be-tested out-of-vocabulary sign language word can be queried to obtain the recorded sign language word corresponding to the to-be-tested out-of-vocabulary sign language word according to the mapping table, if yes, outputting the hit recorded sign language word, if no, executing step (b); (b) judging whether the format of the to-be-tested out-of-vocabulary sign language word is hit by the special style according to the matching mechanism, i.e. judging whether the to-be-tested out-of-vocabulary sign language word is a special style out-of-vocabulary sign language word, if yes, outputting the mapping result corresponding to the special style out-of-vocabulary sign language word according to the matching mechanism, if no, executing step (c); (c) retrieving multiple recorded sign language words for the to-be-tested out-of-vocabulary sign language word according to each word vector index information, and judging the semantic similarity between the to-be-tested out-of-vocabulary sign language word and the recorded sign language words, if the maximum semantic similarity is higher than a threshold, outputting the recorded sign language word corresponding to the maximum semantic similarity, if no, executing step (d); (d) extracting the initial letters of the pinyin English of each character in the to-be-tested out-of-vocabulary sign language word and outputting.

2. The method of claim 1, wherein, The index information of the recorded sign language words is constructed based on the HNSW algorithm, comprising: Multiple word vectors of each recorded sign language word are obtained by using multiple word vector models; word vector index information is generated based on the word vectors generated by each word vector model and the HNSW algorithm, thereby generating multiple word vector index information; the word vectors of the recorded sign language words that are most similar in semantics to the word vector of the out-of-vocabulary word are found by using each word vector index information respectively; the effect of each word vector index information in taking the most similar recorded sign language word in semantics is manually judged and a corresponding weight is given.

3. The method of claim 1, wherein, The index information includes a plurality of word vector index information, each word vector index information is used to retrieve a plurality of recorded sign language words for the to-be-tested unregistered sign language word, and the semantic similarity between the to-be-tested unregistered sign language word and the recorded sign language words is calculated, including: The semantic similarity between the word vector of the to-be-tested unregistered word and the word vector of the recorded sign language word is calculated according to each word vector index information, and the recorded sign language words corresponding to the plurality of word vector pairs with the highest semantic similarity generated by each index information are extracted respectively; The semantic similarity of the plurality of recorded sign language words corresponding to each word vector index information is weighted according to the weight of each word vector index information, and the weighted semantic similarity is obtained.

4. The method of claim 1 or 2, wherein, The semantic similarity is determined according to the Euclidean distance between the word vector of the unregistered word and the word vector of the recorded sign language word.

5. A sign language out-of-vocabulary sign mapping system, comprising: It includes: An index information construction module for constructing index information of recorded sign language words based on the HNSW algorithm; A mapping table construction module for constructing a mapping table of high-frequency unregistered sign language words to recorded sign language words based on the index information of recorded sign language words, specifically including: finding the plurality of word vectors of recorded sign language words that are most similar in semantics to the word vector of the high-frequency unregistered sign language word according to each word vector index information, and respectively acquiring the sign language words corresponding to the word vectors of each recorded sign language word; according to the weight of each word vector index information, the plurality of recorded sign language words are weighted and scored, and the plurality of recorded sign language words are arranged from high to low according to the scores, and the one that is most similar in semantics to the unregistered sign language word is manually judged from the plurality of sign language words, thereby constructing a mapping table of high-frequency unregistered words to recorded sign language words; A matching mechanism construction module for establishing a matching mechanism of special style unregistered sign language words to recorded sign language words based on rules, specifically including: when the special style unregistered sign language word is a word composed of numbers, the numbers are converted into recorded number word combinations; when the special style unregistered sign language word is a word composed of numbers and units, the numbers are converted into recorded number word combinations, and the units are kept; when the special style unregistered sign language word is a word composed of numbers and English or pure English, the numbers and English or English are outputted in character granularity. The mapping module is used to realize the mapping of the unregistered sign language word based on the mapping table, the matching mechanism and the index information, and comprises the following steps: (a) judging whether the unregistered sign language word to be tested is hit by the mapping table, i.e. whether the unregistered sign language word to be tested can be obtained according to the mapping table, if yes, output the hit registered sign language word, if no, execute step (b); (b) judging whether the format of the unregistered sign language word to be tested is hit by the special style, i.e. judging whether the unregistered sign language word to be tested is a special style unregistered sign language word, if yes, output the mapping result of the special style unregistered sign language word according to the matching mechanism, if no, execute step (c); (c) retrieving a plurality of registered sign language words for the unregistered sign language word to be tested according to the index information of each word vector, and judging the semantic similarity between the unregistered sign language word to be tested and the registered sign language words, if the maximum semantic similarity is higher than a threshold, output the registered sign language word corresponding to the maximum semantic similarity, if no, execute step (d); and (d) extracting the English initial letter of the pinyin of each character in the unregistered sign language word to be tested and outputting.

6. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, The memory has the jewelry with labels, and the processor executes the computer program to realize the steps of the sign language unregistered sign language word mapping method in any one of claims 1-4.

7. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to realize the steps of the sign language unregistered sign language word mapping method in any one of claims 1-4.

Images