Sign language recognition model training method and device

Abstract

The embodiments of this specification provide a sign language recognition model training method and device, wherein the sign language recognition model training method includes: determining a combined text based on at least two sample words obtained by splitting a sample text; determining a first sign language video corresponding to a first sample word of the at least two sample words and a second sign language video corresponding to the combined text based on the sign language actions corresponding to the at least two sample words, wherein the first sample word is one of the at least two sample words; using the first sample word as a first sample label of the first sign language video, and using the combined text as a second sample label of the second sign language video; training a sign language recognition model based on the first sign language video, the second sign language video, the first sample label, and the second sample label until the sign language recognition model that meets the training stop condition is obtained.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of model training technology, and in particular to a sign language recognition model training method. Background Art

[0002] Sign language is a specialized language that relies on movement and vision to communicate. It is commonly used by people with hearing impairments or muteness to communicate with the nonverbal community. Sign language uses gestures to mimic movements, mimicking images or syllables to convey meaning or words.

[0003] However, sign language is a vast and complex language system that cannot be matched one-to-one with the natural language used by normal people. When converting natural language into sign language, sign language lacks fixed grammatical rules, and the way people sign the same sentence varies greatly. This results in a limited supply of labeled sign language video data, and the processing cost of sign language video data is high. Therefore, an effective technical solution is urgently needed to address these issues. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a sign language recognition model training method. One or more embodiments of this specification also relate to a sign language recognition model training device, a sign language recognition method, a sign language recognition device, a computing device, a computer-readable storage medium, and a computer program to address technical deficiencies in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a sign language recognition model training method is provided, comprising:

[0006] Determining a combined text based on at least two sample words obtained by splitting the sample text;

[0007] determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word of the at least two sample words, and a second sign language video corresponding to the combined text, wherein the first sample word is one of the at least two sample words;

[0008] Using the first sample word as a first sample label for the first sign language video, and using the combined text as a second sample label for the second sign language video;

[0009] The sign language recognition model is trained according to the first sign language video, the second sign language video, the first sample label, and the second sample label until the sign language recognition model that meets the training stop condition is obtained.

[0010] According to a second aspect of the embodiments of this specification, a sign language recognition model training device is provided, comprising:

[0011] A first determining module is configured to determine a combined text based on at least two sample words obtained by splitting the sample text;

[0012] a second determining module configured to determine, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word of the at least two sample words, and a second sign language video corresponding to the combined text, wherein the first sample word is one of the at least two sample words;

[0013] a labeling module configured to use the first sample word as a first sample label of the first sign language video, and use the combined text as a second sample label of the second sign language video;

[0014] The training module is configured to train the sign language recognition model according to the first sign language video, the second sign language video, the first sample label and the second sample label until the sign language recognition model that meets the training stop condition is obtained.

[0015] According to a third aspect of the embodiments of this specification, a sign language recognition method is provided, including:

[0016] Identify sign language movements;

[0017] Inputting the sign language action into a sign language recognition model, and splitting and recognizing the sign language action using the sign language recognition model;

[0018] Obtaining a recognition result output by the sign language recognition model.

[0019] According to a fourth aspect of the embodiments of this specification, a sign language recognition device is provided, comprising:

[0020] a determination module configured to determine a sign language action;

[0021] An input module is configured to input the sign language action into a sign language recognition model, and to separate and recognize the sign language action using the sign language recognition model;

[0022] The acquisition module is configured to obtain the recognition result output by the sign language recognition model.

[0023] According to a fifth aspect of the embodiments of this specification, there is provided a computing device, including:

[0024] memory and processor;

[0025] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned sign language recognition model training method or sign language recognition method are implemented.

[0026] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned sign language recognition model training method or sign language recognition method.

[0027] According to a seventh aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned sign language recognition model training method or sign language recognition method.

[0028] One embodiment of the present specification provides a method for training a sign language recognition model, which comprises determining a combined text based on at least two sample words obtained by splitting a sample text; determining a first sign language video corresponding to a first sample word of the at least two sample words and a second sign language video corresponding to the combined text based on sign language actions corresponding to the at least two sample words, wherein the first sample word is one of the at least two sample words; using the first sample word as a first sample label of the first sign language video, and using the combined text as a second sample label of the second sign language video; and training a sign language recognition model based on the first sign language video, the second sign language video, the first sample label, and the second sample label until the sign language recognition model that meets the training stop condition is obtained.

[0029] The above method splits the sample text, collects the sign language movements of each word after the split, and combines each word to obtain a combined text. According to the sign language movement of each word, the first sign language video corresponding to each word and the second sign language video corresponding to the combined text can be determined, thereby increasing the training samples, taking into account the impact of different word combinations on sign language movements, and constructing corresponding sample labels for each type of sign language movement, increasing the number of sign language video data with labels, reducing the processing cost of sign language video data, and further improving the training efficiency of the sign language recognition model. Due to the increase in sample types, the trained sign language recognition model can also have higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic diagram of a specific application scenario of a sign language recognition model training method provided by an embodiment of this specification;

[0031] Figure 2 This is a flowchart of a sign language recognition model training method provided by one embodiment of this specification;

[0032] Figure 3 This is a flowchart of a processing process of a sign language recognition model training method provided by one embodiment of this specification;

[0033] Figure 4 This is a structural diagram of a sign language recognition model training device provided by one embodiment of this specification;

[0034] Figure 5 is a flow chart of a sign language recognition method provided by one embodiment of this specification;

[0035] Figure 6 This is a schematic diagram of a specific application scenario of a sign language recognition method provided by an embodiment of this specification;

[0036] Figure 7 This is a structural diagram of a sign language recognition device provided by one embodiment of this specification;

[0037] Figure 8 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0038] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0039] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0040] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0041] In practice, sign language is the first language of the hearing-impaired, and they usually communicate with the outside world through sign language. However, without systematic sign language training, hearing people cannot understand the meaning of the sign language, which leads to communication difficulties with the hearing-impaired and affects their communication with the outside world. Therefore, a sign language recognition device or a sign language translation device can be used to recognize and translate sign language, so that hearing people can understand the meaning of the hearing-impaired, thus facilitating communication between the hearing-impaired and the hearing-impaired.

[0042] However, because people in different regions may use different sign gestures for the same word, and individual sign gestures vary, the regional, diverse, and personalized nature of sign language makes it difficult for sign language recognition devices to identify every sign gesture. Furthermore, current sign language video datasets are relatively rare, with few accurately labeled sign language videos.

[0043] Typically, when collecting sign language video data, one can use a universal sign language dictionary and standardized sign language movements to collect sign language video data. However, due to the large number of fused words in the dictionary, a large amount of sign language movement data is repeated during the data collection process, increasing data processing costs. Alternatively, one can record the sign language video data corresponding to a given vocabulary. However, this method is not conducive to data management, has high data processing costs, low data collection efficiency, and poor data collection quality. Therefore, an effective technical solution is urgently needed to solve the above problems.

[0044] In this specification, a sign language recognition model training method is provided. This specification also relates to a sign language recognition model training device, a sign language recognition method, a sign language recognition device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0045] See also Figure 1 , Figure 1 A schematic diagram of a specific application scenario of a sign language recognition model training method provided according to an embodiment of this specification is shown.

[0046] Figure 1 The system includes a terminal and a server, wherein the terminal and the server are in communication connection. The terminal can be a mobile terminal, such as a mobile phone, or a computer terminal. The server can be used to train the sign language recognition model.

[0047] During specific implementation, the terminal can split the sample text to obtain at least two sample words, and combine the at least two sample words to obtain a combined text; thereafter, the first sign language video corresponding to the first sample word of the at least two sample words and the second sign language video corresponding to the combined text can be determined. Specifically, the at least two sample words and the combined text can be displayed to the user through the terminal, and a task of recording a video is generated on the terminal. The user clicks on one of the recording tasks on the terminal interface and clicks to start recording. Specifically, the user can record the first sign language video corresponding to each sample word and the second sign language video corresponding to the combined text; the terminal sends the video recorded by the user to the server, and the server can jointly train the sign language recognition model based on the first sign language video and at least two sample words, the second sign language video and the combined text to obtain a trained sign language recognition model. The training samples and training labels for the sign language recognition model training are increased, the acquisition quality and efficiency of the sign language video data are improved, and the recognition accuracy of the trained sign language recognition model is further increased. It is understandable that the process of splitting the sample text to obtain at least two sample words and combining the at least two sample words to obtain the combined text can also be implemented by the server, and the embodiments of this specification are not limited thereto.

[0048] It should be noted that the first sign language video and the second sign language video can be obtained by recording by the user, by recording by a virtual character, by using a motion recognition model and a motion editing model, or by cutting existing sign language motion videos. This specification does not limit this.

[0049] like Figure 1 In the example, taking the sample text "the weather is very cold", the sample text can be split into three sample words "weather", "very", and "cold", and these three sample words can be combined to obtain the combined text "cold, weather, very". Determine the first sign language video corresponding to the sample word "weather" and the second sign language video corresponding to the combined text. Specifically, these three sample words and the combined text can be generated on the terminal to generate a recording task and displayed to the user. The user can record the first sign language video corresponding to the sample word "weather" or the second sign language video corresponding to the combined text "cold, weather, very". The terminal sends the first sign language video and the second sign language video to the server. The server can jointly train the sign language recognition model based on the first sign language video and the sample word "weather", the second sign language video and the combined text "cold, weather, very", and obtain a trained sign language recognition model.

[0050] Understandably, although Figure 1Not shown in the figure, the first sign language videos corresponding to the determined sample words may also include the first sign language videos corresponding to the sample word “very” and the first sign language videos corresponding to the sample word “cold”.

[0051] Through the above-mentioned method of training the sign language recognition model, the training samples and training labels of the sign language recognition model are increased, the acquisition quality and efficiency of the sign language video data are improved, and the recognition accuracy of the trained sign language recognition model is further increased.

[0052] See also Figure 2 , Figure 2 A flowchart of a sign language recognition model training method provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0053] Step 202: Determine a combined text based on at least two sample words obtained by splitting the sample text.

[0054] Among them, the sample text can be understood as the text for which sign language video needs to be generated, which can be a complete sentence said by a normal person when communicating, such as "The weather is very good today", "This watermelon is very cheap, do you want to buy some", etc.; splitting the sample text can be understood as splitting a whole sentence said by a normal person when communicating into at least two words, and these two words are at least two sample words obtained by splitting the sample text, for example, the sample text "The weather is very good today" can be split into "today", "weather", "very", and "good", and the four words obtained are sample words; combined text can be understood as the text obtained by combining at least two sample words, such as "today", "weather", "very", and "good" to obtain "today, very, good, weather".

[0055] Specifically, in order to prevent the sample words from being incorrectly separated in the process of splitting the sample text into sample words and then recombining them to obtain the combined text, that is, one sample word is incorrectly combined into another sample word, for example, when combining "today", "weather", "very", and "good", due to the incorrect separation of one of the words, the combined text is caused to be "Today the weather is very good", a separator can be set in the combined text. The combined text is composed of at least two sample words, so the separator can be set between each sample word, for example, the separator can be ",", " / ", "-", and the combined text can be "Today, very, good, weather", "Today-very-good-weather", etc.

[0056] Based on this, the complete words for which a sign language video needs to be generated can be split to obtain at least two words, and the at least two words can be recombined to determine a combined text.

[0057] In addition, after determining the combined text, the sample words can be input into the action recognition model to obtain the sign language actions corresponding to the sample words. The specific implementation method is as follows:

[0058] The at least two sample words are input into a motion recognition model to obtain sign language actions corresponding to the at least two sample words.

[0059] Among them, the action recognition model can be understood as a model that can convert text information into action information. In actual applications, any model that can achieve this function can be selected to complete the conversion from sample words to sign language actions. The embodiments of this specification do not limit this; sign language actions can be understood as action text.

[0060] It should be noted that inputting at least two sample words into the action recognition model can be understood as inputting each of the at least two sample words into the action recognition model, and then obtaining the sign language actions corresponding to the at least two sample words can be understood as obtaining the sign language actions corresponding to each of the at least two sample words.

[0061] Then, when determining the first sign language video corresponding to the first sample word among the at least two sample words based on the sign language actions corresponding to the at least two sample words, this can also be accomplished through a model that can implement the corresponding function, specifically:

[0062] The sign language actions corresponding to the at least two sample words are input into a motion editing model to obtain a first sign language video corresponding to a first sample word among the at least two sample words.

[0063] Among them, the sign language actions corresponding to at least two sample words can be understood as the sign language actions corresponding to each sample word in the at least two sample words; then inputting the sign language actions corresponding to at least two sample words into the action editing model can be understood as inputting the sign language actions corresponding to each sample word into the action editing model. Since there are at least two sample words, the inputs to the action editing model should also be at least two sign language actions; further, the first sign language video corresponding to the first sample word in the at least two sample words obtained can also be understood as the first sign language video corresponding to each sample word in the at least two sample words. For example, among 3 sample words, the sign language actions corresponding to each sample word in these 3 sample words can be determined. Then the sign language actions determined at this time should also be 3, and the first sign language videos finally obtained should also be 3; the action editing model can be understood as a model that can convert at least two sign language actions into a first sign language video.

[0064] Based on this, after splitting the sample text to obtain at least two sample words, the at least two sample words can be input into the action recognition model, the at least two sample words can be converted into at least two sign language actions, and then the at least two sign language actions can be input into the action editing model. The at least two sign language actions can be edited by the action editing model to generate the first sign language video corresponding to each sample word.

[0065] For example, the sample text "How much is watermelon?" is split into three sample words: "watermelon," "how much," and "money." These three sample words are recombined to obtain the combined text "watermelon, money, how much." Based on the sample word "watermelon," the sign language gesture corresponding to "watermelon" is obtained, and based on the sign language gesture corresponding to "watermelon," the first sign language video corresponding to "watermelon" is further obtained. Based on the sample word "how much," the sign language gesture corresponding to "how much" is obtained, and based on the sign language gesture corresponding to "how much," the first sign language video corresponding to "how much" is further obtained. Based on the sample word "money," the sign language gesture corresponding to "money" is obtained, and based on the sign language gesture corresponding to "money," the first sign language video corresponding to "money" is further obtained.

[0066] In summary, by obtaining the sign language action corresponding to each sample word, we can further obtain the first sign language video corresponding to each sample word, providing a data basis for subsequently obtaining the second sign language video corresponding to the combined text, and can increase the amount of training data for training the sign language recognition model, further improving the training quality.

[0067] In practical applications, in order to ensure the quantity and quality of training samples when training a sign language recognition model, when determining a combined text based on at least two sample words obtained by splitting the sample text, the at least two sample words can be sorted and reorganized according to a preset sorting rule to obtain a combined text. The specific implementation method is as follows:

[0068] Receive sample text uploaded by users;

[0069] Splitting the sample text to obtain at least two sample words corresponding to the sample text;

[0070] The at least two sample words are sorted according to a preset sorting rule to obtain a combined text.

[0071] Among them, the sample text can be uploaded by the user through the terminal, or it can be the collected voice communication between users, and the voice communication is converted into sample text; the preset sorting rules can be understood as the pre-set sorting rules between sample words, such as sequential sorting rules and / or random sorting rules, etc.

[0072] Based on this, a sample text uploaded by a user through a terminal can be received, and the sample text can be split according to the semantics of the sample text to obtain at least two sample words. The at least two sample words can be sorted according to the pre-set sorting rules between the sample words to obtain a combined text.

[0073] It should be noted that the preset sorting rules may include multiple sorting rules, which can increase the amount of training data and also consider the impact of different word orders on sign language movements, thereby improving the training quality of the sign language recognition model.

[0074] Continuing with the above example, for the sample text "How much is watermelon", split it into three sample words "watermelon", "how much", and "money". According to the preset sorting rules, the three sample words are combined into "watermelon, money, how much", "watermelon, how much, money", "money, how much, watermelon", etc.

[0075] In summary, by sorting at least two sample words according to a variety of preset sorting rules, a variety of combined texts can be obtained, which provides a data basis for subsequently obtaining the second sign language video corresponding to the combined text, increases the amount of training data when training the sign language recognition model, saves data management and collection costs, and takes into account the impact of different word orders on sign language movements, further improving the training quality of the sign language recognition model, thereby facilitating the sign language recognition model to split and recognize sign language movements in subsequent applications.

[0076] In specific implementation, the preset sorting rules may include sequential sorting rules and / or random sorting rules. At least two sample words may be sorted according to a plurality of preset sorting rules to obtain a combined text. The specific implementation is as follows:

[0077] Sorting the at least two sample words in sequence according to a sequence sorting rule to obtain a combined text; and / or,

[0078] Randomly sorting the at least two sample words according to a random sorting rule to obtain a combined text;

[0079] The sequential sorting rule is a sorting rule for the at least two sample words that are not separated in the sample text.

[0080] Specifically, the sequential sorting rule can be understood as the order of at least two sample words from front to back in the unsplit sample text. For example, for the sample text "The weather is very good today", the split sample words are "today", "weather", "very", and "good". The sequential sorting rule is to sort them in the order of "today", "weather", "very", and "good". The combined text obtained at this time is "today, weather, very, good"; the sequential sorting rule can also be understood as the order of at least two sample words from back to front in the unsplit sample text. For example, for the sample text "The weather is very good today", the sequential sorting rule can be to sort them in the order of "good", "very", "weather", and "today". The combined text obtained at this time is "good, very, weather, today".

[0081] Then, the random sorting rule can be understood as a rule for randomly sorting and combining at least two sample words. For example, for the sample text "The weather is very good today", the sample words "today", "weather", "very" and "good" are randomly sorted, and the resulting combined text can be, for example, "today, very, weather, good", or "weather, good, very, today".

[0082] Based on this, at least two sample words may be sequentially or randomly sorted according to a sequential sorting rule and / or a random sorting rule to obtain combined texts with different sorting orders.

[0083] In summary, by sorting at least two sample words according to sequential sorting rules and / or random sorting rules, the impact of different word orders on sign language movements can be taken into account, which facilitates the subsequent generation of a second sign language video corresponding to the combined text under different word orders. That is, it can increase the richness of training data, free sign language recognition from the limitations of scripts and fixed combinations, save data collection costs while ensuring diversity and recognition effects, and thus improve training quality.

[0084] Step 204: Determine a first sign language video corresponding to a first sample word among the at least two sample words and a second sign language video corresponding to the combined text based on the sign language actions corresponding to the at least two sample words, wherein the first sample word is one of the at least two sample words.

[0085] Specifically, after obtaining the combined text according to at least two sample words, the first sign language video and the second sign language video may be determined according to the sign language actions corresponding to the at least two sample words.

[0086] Among them, the first sample word can be understood as a sample word among at least two sample words; the first sign language video corresponding to the first sample word among at least two sample words can be understood as the first sign language video corresponding to the sample word among at least two sample words. In other words, there is a one-to-one correspondence between a sample word and a first sign language video, so at least two sample words also correspond to at least two first sign language videos.

[0087] It should be noted that, taking three sample words as an example, the three sample words are "today," "very," and "cold," the first sample word can be "today," "very," or "cold." Specifically, based on the sign language movements corresponding to the at least two sample words, each first sign language video corresponding to each first sample word in the at least two sample words can be determined. For example, the first sign language video corresponding to the first sample word "today" can be determined, the first sign language video corresponding to the first sample word "very" can be determined, and the first sign language video corresponding to the first sample word "cold" can be determined.

[0088] Based on this, the first sign language video corresponding to each sample word and the second sign language video corresponding to the combined text can be determined according to the sign language action corresponding to each sample word in the at least two sample words.

[0089] Continuing with the above example, for the sample text "How much does watermelon cost", split it into three sample words "watermelon", "how much", and "money". According to the preset sorting rules, the three sample words are combined into a combined text "watermelon, money, how much". According to the sign language movements corresponding to "watermelon", the sign language movements corresponding to "how much", and the sign language movements corresponding to "money", determine the first sign language video corresponding to "watermelon", the first sign language video corresponding to "how much", the first sign language video corresponding to "money", and the second sign language video corresponding to "watermelon, money, how much".

[0090] In actual applications, when determining the second sign language video corresponding to the combined text, each first sign language video corresponding to each first sample word in the combined text may be spliced ​​together to obtain the second sign language video. The specific implementation method is as follows:

[0091] Determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word among the at least two sample words;

[0092] According to the order of the at least two sample words in the combined text, the first sign language video corresponding to the first sample word of the at least two sample words in the combined text is spliced ​​to obtain the second sign language video corresponding to the combined text.

[0093] Specifically, since the combined text is obtained by combining sample words, each first sign language video corresponding to each first sample word can be spliced ​​in the order of the sample words in the combined text to obtain the second sign language video corresponding to the combined text.

[0094] Continuing with the above example, for the sample text "How much is watermelon", it is split into three sample words "watermelon", "how much", and "money". According to the preset sorting rules, the three sample words are combined into a combined text "watermelon, money, how much". The order of the sample words in the combined text is "watermelon" to "money" to "how much". The first sign language video corresponding to "watermelon", the first sign language video corresponding to "money", and the first sign language video corresponding to "how much" can be spliced ​​in this order to obtain the second sign language video corresponding to the combined text "watermelon, money, how much".

[0095] It should be noted that, based on the sign language actions corresponding to at least two sample words, the specific implementation method of determining each first sign language video corresponding to the first sample word in at least two sample words can be referred to the specific implementation method of determining the first sign language video in the above embodiment, which will not be repeated here.

[0096] In summary, by splicing the first sign language videos corresponding to the existing sample words to obtain the second sign language video, the recording cost of the sign language video can be reduced, and more sign language action combination videos can be obtained, thereby reducing the training cost of the sign language recognition model, increasing the training data, and improving the generalization ability of the sign language recognition model in the application process, thereby reducing the cost of scene migration.

[0097] In addition, when determining the first sign language video corresponding to the first sample word of the at least two sample words based on the sign language movements corresponding to the at least two sample words, the user may also manually record the video to integrate the sign language movement habits of different users. The specific implementation method is as follows:

[0098] Displaying the at least two sample words and the sign language actions corresponding to the at least two sample words to the user respectively;

[0099] A first sign language video corresponding to a first sample word among the at least two sample words recorded by the user is received.

[0100] Specifically, the sample words to be recorded and the sign language movements corresponding to the sample words can be displayed to the user through a mobile terminal or a computer terminal. The user can record according to the sign language movements and upload the recorded first sign language video.

[0101] In one embodiment of the present specification, a virtual character may be used to record the first sign language video according to the sign language movements corresponding to the sample words, and this embodiment of the present specification does not limit this.

[0102] In summary, by having users record their first sign language videos, we can integrate their sign language movement habits, enhance the diversity and individuality of their first sign language videos, and provide sufficient sample data for subsequently improving the generalization ability of the sign language recognition model.

[0103] Furthermore, in sign language, a word may have multiple sign language gestures. Accordingly, a sign language gesture may also contain multiple word meanings. In order to ensure the comprehensiveness and diversity of the training data, when determining the first sign language video corresponding to the first sample word of the at least two sample words based on the sign language gestures corresponding to the at least two sample words, different sign language gestures corresponding to the sample words may also be recorded. The specific implementation method is as follows:

[0104] Determining at least two sign language actions corresponding to a first sample word among the at least two sample words;

[0105] At least two first sign language videos corresponding to the first sample words are determined according to the at least two sign language actions.

[0106] Taking the sample word "watermelon" as an example, the sign language action used to represent "watermelon" can be to use both hands to represent a circle, or to use both hands to represent a circle and place it next to the ear, and then make a clapping motion with the hands. In this case, the corresponding first sign language video can be recorded for each sign language action corresponding to the sample word to ensure the comprehensiveness of the training data, thereby improving the quality of sign language recognition model training.

[0107] In summary, by collecting different sign language movements corresponding to a sample word, we ensure that subtle differences in sign language movements for the same sample word due to personal habits are included, thus ensuring data diversity.

[0108] In actual applications, when performing sign language actions for a sample word or a combined text composed of sample words, due to different personal habits, it is possible to perform sign language actions continuously or omit certain sign language actions. Therefore, when determining the second sign language video corresponding to the combined text, it is also possible to first combine the sign language actions corresponding to the sample words in the combined text, and then record according to the combined sign language actions to obtain the second sign language video. The specific implementation method is as follows:

[0109] Determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word of the at least two sample words and a second sign language video corresponding to the combined text, includes:

[0110] Determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word among the at least two sample words;

[0111] splicing the sign language actions corresponding to the at least two sample words in the combined text according to the order of the at least two sample words in the combined text to obtain a spliced ​​sign language action corresponding to the combined text;

[0112] According to the spliced ​​sign language action, a second sign language video corresponding to the combined text is determined.

[0113] The spliced ​​sign language action can be understood as the sign language action obtained by combining the action texts corresponding to each sample word in sequence.

[0114] Based on this, the sign language actions corresponding to each sample word can be combined and spliced ​​according to the order of the sample words in the combined text to obtain a spliced ​​sign language action, and then the second sign language video corresponding to the combined text can be determined based on the spliced ​​sign language action.

[0115] Specifically, when determining the second sign language video corresponding to the combined text based on the spliced ​​sign language action, the spliced ​​sign language action and the combined text can be displayed to the user so that the user can record it. The specific implementation method is as follows:

[0116] Displaying the combined text and the spliced ​​sign language action corresponding to the combined text to the user;

[0117] Receive a second sign language video corresponding to the combined text recorded by the user.

[0118] In addition, when determining the second sign language video corresponding to the combined text according to the spliced ​​sign language action, the second sign language video can also be obtained through the action editing model, and the specific implementation method is as follows:

[0119] The spliced ​​sign language action is input into an action editing model to obtain a second sign language video corresponding to the combined text.

[0120] It can be understood that the process of obtaining the second sign language video is similar to the process of obtaining the first sign language video, and will not be repeated here.

[0121] In summary, by having users record a second sign language video, the diversity of training data is increased, so that the sign language recognition model can learn the user's personal habits, continuous typing, omissions, etc., thereby improving the training quality of the sign language recognition model, making the recognition accuracy of subsequent sign language recognition models higher, and achieving comprehensive recognition of the sign language recognition model.

[0122] Step 206: Use the first sample word as a first sample label of the first sign language video, and use the combined text as a second sample label of the second sign language video.

[0123] Specifically, after determining the first sign language video and the second sign language video, the first sample word can be used as the first sample label of the first sign language video, and the combined text can be used as the second sample label of the second sign language video, thereby training the sign language recognition model. It should be noted that in the case of determining each first sign language video corresponding to each first sample word of the at least two sample words, each first sample word can be used as the first sample label of each first sign language video.

[0124] Here, taking the first sample word as the first sample label of the first sign language video can be understood as taking each first sample word as the first sample label of each first sign language video corresponding to the first sample word.

[0125] Continuing with the above example, for the sample text "How much does watermelon cost?", split it into three sample words: "watermelon," "how much," and "money." Based on the preset sorting rules, combine the three sample words into the combined text "watermelon, money, how much." Based on the sign language gestures corresponding to "watermelon," "how much," and "money," determine the first sign language video corresponding to "watermelon," the first sign language video corresponding to "how much," the first sign language video corresponding to "money," and the second sign language video corresponding to "watermelon, money, how much." "Watermelon" is used as the first sample label for the first sign language video corresponding to "watermelon," "how much" is used as the first sample label for the first sign language video corresponding to "how much," and "money" is used as the first sample label for the first sign language video corresponding to "money." "Watermelon, money, how much" is used as the second sample label for the second sign language video corresponding to "watermelon, money, how much."

[0126] Step 208: Train the sign language recognition model based on the first sign language video, the second sign language video, the first sample label, and the second sample label until the sign language recognition model that meets the training stop condition is obtained.

[0127] Specifically, the sign language recognition model may be jointly trained based on the first sign language video and the first sample label, the second sign language video and the second sample label, until a sign language recognition model that meets the training stop condition is obtained.

[0128] The training stop condition can be understood as the training reaching a preset round or the model loss value reaching a preset threshold.

[0129] In summary, the above method splits the sample text, collects the sign language actions of each word after splitting, combines each word to obtain a combined text, and determines the first sign language video corresponding to each word and the second sign language video corresponding to the combined text based on the sign language actions of each word. This increases the training samples, takes into account the influence of different word combination methods on sign language actions, constructs corresponding sample labels for each type of sign language action, increases the sign language video data with labels, reduces the processing cost of sign language video data, further improves the training efficiency of the sign language recognition model, and further enables the trained sign language recognition model to have high accuracy due to the increase in sample types.

[0130] The following combines the attached Figure 3 , taking the application of the sign language recognition model training method provided in this specification in model training as an example, further explains the sign language recognition model training method. Among them, Figure 3 shows the processing flowchart of a sign language recognition model training method provided in an embodiment of this specification, which specifically includes the following steps.

[0131] Step 302: Receive the sample text uploaded by the user.

[0132] The sample text received from the user is "I want to eat watermelon".

[0133] Step 304: Split the sample text to obtain sample words.

[0134] Split "I want to eat watermelon" to obtain 4 sample words: "I", "want", "eat", "watermelon".

[0135] Step 306: Recombine the sample words according to a preset sorting rule to obtain a combined text.

[0136] Randomly sort and recombine the 4 sample words to obtain the combined text: "want, watermelon, I, eat".

[0137] Step 308: Convert each sample word into the corresponding sign language action.

[0138] Input "want" into the action recognition model and convert it into the sign language action text corresponding to "want"; input "I" into the action recognition model and convert it into the sign language action text corresponding to "I"; input "eat" into the action recognition model and convert it into the sign language action text corresponding to "eat"; input "watermelon" into the action recognition model and convert it into the sign language action text corresponding to "watermelon".

[0139] Step 310: Record the first sign language video corresponding to each sample word.

[0140] According to the sign language action text corresponding to "think", record the first sign language video corresponding to "think"; according to the sign language action text corresponding to "I", record the first sign language video corresponding to "I"; according to the sign language action text corresponding to "eat", record the first sign language video corresponding to "eat"; according to the sign language action text corresponding to "watermelon", record the first sign language video corresponding to "watermelon".

[0141] Step 312: Record a second sign language video corresponding to the combined text.

[0142] According to the sign language action text corresponding to "think", the sign language action text corresponding to "I", the sign language action text corresponding to "eat", and the sign language action text corresponding to "watermelon", the spliced ​​sign language action text corresponding to the combined text "think, watermelon, I, eat" is determined, and according to the spliced ​​sign language action text, a second sign language video corresponding to the combined text "think, watermelon, I, eat" is recorded.

[0143] Step 314: Train the sign language recognition model based on the first sign language video and the sample words, and based on the second sign language video and the combined text, until a sign language recognition model that meets the training stop condition is obtained.

[0144] In summary, the above method splits the sample text, collects the sign language movements of each word after the split, and combines each word to obtain a combined text. According to the sign language movement of each word, the first sign language video corresponding to each word and the second sign language video corresponding to the combined text can be determined, which increases the training samples, takes into account the impact of different word combinations on sign language movements, and constructs corresponding sample labels for each type of sign language movement, increases the number of sign language video data with labels, reduces the processing cost of sign language video data, and further improves the training efficiency of the sign language recognition model. Due to the increase in sample types, the trained sign language recognition model can also have higher accuracy.

[0145] Corresponding to the above method embodiment, this specification also provides an embodiment of a sign language recognition model training device, Figure 4 FIG1 shows a schematic diagram of a structure of a sign language recognition model training device provided by an embodiment of this specification. Figure 4 As shown, the device includes:

[0146] A first determining module 402 is configured to determine a combined text based on at least two sample words obtained by splitting the sample text;

[0147] A second determining module 404 is configured to determine, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word of the at least two sample words, and a second sign language video corresponding to the combined text, wherein the first sample word is one of the at least two sample words;

[0148] a labeling module 406 configured to use the first sample word as a first sample label for the first sign language video, and use the combined text as a second sample label for the second sign language video;

[0149] The training module 408 is configured to train the sign language recognition model according to the first sign language video, the second sign language video, the first sample label and the second sample label until the sign language recognition model that meets the training stop condition is obtained.

[0150] In an optional embodiment, the second determining module 404 is further configured to:

[0151] Determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word among the at least two sample words;

[0152] According to the order of the at least two sample words in the combined text, the first sign language video corresponding to the first sample word of the at least two sample words in the combined text is spliced ​​to obtain the second sign language video corresponding to the combined text.

[0153] In an optional embodiment, the second determining module 404 is further configured to:

[0154] Displaying the at least two sample words and the sign language actions corresponding to the at least two sample words to the user respectively;

[0155] A first sign language video corresponding to a first sample word among the at least two sample words recorded by the user is received.

[0156] In an optional embodiment, the second determining module 404 is further configured to:

[0157] Determining at least two sign language actions corresponding to a first sample word among the at least two sample words;

[0158] At least two first sign language videos corresponding to the first sample words are determined according to the at least two sign language actions.

[0159] In an optional embodiment, the second determining module 404 is further configured to:

[0160] Determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word among the at least two sample words;

[0161] splicing the sign language actions corresponding to the at least two sample words in the combined text according to the order of the at least two sample words in the combined text to obtain a spliced ​​sign language action corresponding to the combined text;

[0162] According to the spliced ​​sign language action, a second sign language video corresponding to the combined text is determined.

[0163] In an optional embodiment, the second determining module 404 is further configured to:

[0164] Displaying the combined text and the spliced ​​sign language action corresponding to the combined text to the user;

[0165] Receive a second sign language video corresponding to the combined text recorded by the user.

[0166] In an optional embodiment, the second determining module 404 is further configured to:

[0167] The spliced ​​sign language action is input into an action editing model to obtain a second sign language video corresponding to the combined text.

[0168] In an optional embodiment, the second determining module 404 is further configured to:

[0169] Inputting the at least two sample words into a motion recognition model to obtain sign language actions corresponding to the at least two sample words;

[0170] The sign language actions corresponding to the at least two sample words are input into a motion editing model to obtain a first sign language video corresponding to a first sample word among the at least two sample words.

[0171] In an optional embodiment, the first determining module 402 is further configured to:

[0172] Receive sample text uploaded by users;

[0173] Splitting the sample text to obtain at least two sample words corresponding to the sample text;

[0174] The at least two sample words are sorted according to a preset sorting rule to obtain a combined text.

[0175] In an optional embodiment, the first determining module 402 is further configured to:

[0176] Sorting the at least two sample words in sequence according to a sequence sorting rule to obtain a combined text; and / or,

[0177] Randomly sorting the at least two sample words according to a random sorting rule to obtain a combined text;

[0178] The sequential sorting rule is a sorting rule for the at least two sample words that are not separated in the sample text.

[0179] In summary, the above-mentioned device splits the sample text, collects the sign language movements of each word after the split, and combines each word to obtain a combined text. According to the sign language movement of each word, the first sign language video corresponding to each word and the second sign language video corresponding to the combined text can be determined, thereby increasing the training samples, taking into account the impact of different word combinations on sign language movements, and constructing corresponding sample labels for each type of sign language movement, increasing the number of sign language video data with labels, reducing the processing cost of sign language video data, and further improving the training efficiency of the sign language recognition model. Due to the increase in sample types, the trained sign language recognition model can also have higher accuracy.

[0180] The above is a schematic diagram of a sign language recognition model training device according to this embodiment. It should be noted that the technical solution of this sign language recognition model training device and the technical solution of the aforementioned sign language recognition model training method are based on the same concept. For details not described in detail in the technical solution of the sign language recognition model training device, please refer to the description of the technical solution of the aforementioned sign language recognition model training method.

[0181] See also Figure 5 , Figure 5 A flowchart of a sign language recognition method provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0182] Step 502: Determine the sign language action;

[0183] Step 504: Inputting the sign language action into a sign language recognition model, and using the sign language recognition model to split and recognize the sign language action;

[0184] Step 506: Obtain the recognition result output by the sign language recognition model.

[0185] Specifically, the user's sign language movements can be captured by a camera device, and the sign language movements can be input into a sign language recognition model. The sign language recognition model can split and recognize the sign language movements to obtain the recognition results output by the sign language recognition model. The recognition results are then converted into natural language text, that is, sentences communicated between normal people, and broadcast through voice to complete sign language communication between users.

[0186] In summary, the sign language recognition model trained by the above-mentioned sign language recognition model training method can recognize the captured sign language movements, and can realize the separation and recognition of sign language movements, with a high recognition accuracy rate, and the recognized sign language movements are more comprehensive.

[0187] See also Figure 6 , Figure 6 A schematic diagram of a specific application scenario of a sign language recognition method provided according to an embodiment of this specification is shown.

[0188] Figure 6 The system includes a terminal and a server that communicates with the terminal. The terminal can be a mobile terminal, such as a mobile phone, or a device with a camera and voice playback function. The server is deployed with a sign language recognition model that can be used to recognize sign language movements.

[0189] During specific implementation, the terminal can capture the user's sign language movements and send them to the server. The sign language recognition model deployed on the server can split the sign language movements, and recognize and translate each sign language movement obtained after the split, and finally output the recognition result "I am cold" to the terminal. The terminal can broadcast the recognition result "I am cold" through voice.

[0190] The above sign language recognition method can realize the separation and recognition of sign language movements, and realize sign language communication between users.

[0191] Corresponding to the above method embodiment, this specification also provides an embodiment of a sign language recognition device, Figure 7 FIG1 shows a schematic diagram of a structure of a sign language recognition model training device provided by an embodiment of this specification. Figure 7 As shown, the device includes:

[0192] A determination module 702 is configured to determine a sign language action;

[0193] An input module 704 is configured to input the sign language gesture into a sign language recognition model, and to separate and recognize the sign language gesture using the sign language recognition model;

[0194] The obtaining module 706 is configured to obtain the recognition result output by the sign language recognition model.

[0195] In summary, the sign language recognition model trained by the above-mentioned sign language recognition model training method can recognize the captured sign language movements, and can realize the separation and recognition of sign language movements, with a high recognition accuracy rate, and the recognized sign language movements are more comprehensive.

[0196] The above is a schematic diagram of a sign language recognition device according to this embodiment. It should be noted that the technical solution of this sign language recognition device and the technical solution of the aforementioned sign language recognition method are based on the same concept. For details not described in detail in the technical solution of the sign language recognition device, please refer to the description of the technical solution of the aforementioned sign language recognition method.

[0197] Figure 8 8 shows a block diagram of a computing device 800 according to one embodiment of the present disclosure. Components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0198] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)), whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0199] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 8 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0200] The computing device 800 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 800 can also be a mobile or stationary server.

[0201] The processor 820 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned sign language recognition model training method or sign language recognition method.

[0202] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the aforementioned sign language recognition model training method or sign language recognition method are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the aforementioned sign language recognition model training method or sign language recognition method.

[0203] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned sign language recognition model training method or sign language recognition method.

[0204] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium is based on the same concept as the technical solution of the sign language recognition model training method or the sign language recognition method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the sign language recognition model training method or the sign language recognition method described above.

[0205] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned sign language recognition model training method or sign language recognition method.

[0206] The above is an illustrative solution of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the sign language recognition model training method or the sign language recognition method described above are based on the same concept. For details not described in detail in the technical solution of the computer program, please refer to the description of the technical solution of the sign language recognition model training method or the sign language recognition method described above.

[0207] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0208] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0209] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0210] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0211] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A sign language recognition model training method, comprising: Determining a combined text based on at least two sample words obtained by splitting the sample text; determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word of the at least two sample words, and a second sign language video corresponding to the combined text, wherein the first sample word is one of the at least two sample words; Using the first sample word as a first sample label for the first sign language video, and using the combined text as a second sample label for the second sign language video; The sign language recognition model is trained according to the first sign language video, the second sign language video, the first sample label, and the second sample label until the sign language recognition model that meets the training stop condition is obtained.

2. The method according to claim 1, wherein determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word of the at least two sample words and a second sign language video corresponding to the combined text comprises: Determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word among the at least two sample words; According to the order of the at least two sample words in the combined text, the first sign language video corresponding to the first sample word of the at least two sample words in the combined text is spliced ​​to obtain the second sign language video corresponding to the combined text.

3. The method according to claim 2, wherein determining a first sign language video corresponding to a first sample word among the at least two sample words based on the sign language actions corresponding to the at least two sample words comprises: Displaying the at least two sample words and the sign language actions corresponding to the at least two sample words to the user respectively; A first sign language video corresponding to a first sample word among the at least two sample words recorded by the user is received.

4. The method according to claim 2, wherein determining a first sign language video corresponding to a first sample word among the at least two sample words based on the sign language actions corresponding to the at least two sample words comprises: Determining at least two sign language actions corresponding to a first sample word among the at least two sample words; At least two first sign language videos corresponding to the first sample words are determined according to the at least two sign language actions.

5. The method according to claim 1, wherein determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word of the at least two sample words and a second sign language video corresponding to the combined text comprises: Determining, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word among the at least two sample words; splicing the sign language actions corresponding to the at least two sample words in the combined text according to the order of the at least two sample words in the combined text to obtain a spliced ​​sign language action corresponding to the combined text; According to the spliced ​​sign language action, a second sign language video corresponding to the combined text is determined.

6. The method according to claim 5, wherein determining the second sign language video corresponding to the combined text based on the spliced ​​sign language action comprises: Displaying the combined text and the spliced ​​sign language action corresponding to the combined text to the user; Receive a second sign language video corresponding to the combined text recorded by the user.

7. The method according to claim 5, wherein determining the second sign language video corresponding to the combined text based on the spliced ​​sign language action comprises: The spliced ​​sign language action is input into an action editing model to obtain a second sign language video corresponding to the combined text.

8. The method according to claim 1, after determining the combined text, further comprising: Inputting the at least two sample words into a motion recognition model to obtain sign language actions corresponding to the at least two sample words; Accordingly, determining a first sign language video corresponding to a first sample word among the at least two sample words according to the sign language actions corresponding to the at least two sample words includes: The sign language actions corresponding to the at least two sample words are input into a motion editing model to obtain a first sign language video corresponding to a first sample word among the at least two sample words.

9. The method according to claim 1, wherein determining a combined text based on at least two sample words obtained by splitting the sample text comprises: Receive sample text uploaded by users; Splitting the sample text to obtain at least two sample words corresponding to the sample text; The at least two sample words are sorted according to a preset sorting rule to obtain a combined text.

10. The method according to claim 9, wherein the step of sorting the at least two sample words according to a preset sorting rule to obtain a combined text comprises: Sorting the at least two sample words in sequence according to a sequence sorting rule to obtain a combined text; and / or, Randomly sorting the at least two sample words according to a random sorting rule to obtain a combined text; The sequential sorting rule is a sorting rule for the at least two sample words that are not separated in the sample text.

11. A sign language recognition method, comprising: Identify sign language movements; Inputting the sign language action into the sign language recognition model according to claims 1 to 10, and splitting and recognizing the sign language action using the sign language recognition model; Obtaining a recognition result output by the sign language recognition model.

12. A sign language recognition model training device, comprising: A first determining module is configured to determine a combined text based on at least two sample words obtained by splitting the sample text; a second determining module configured to determine, based on the sign language actions corresponding to the at least two sample words, a first sign language video corresponding to a first sample word of the at least two sample words, and a second sign language video corresponding to the combined text, wherein the first sample word is one of the at least two sample words; a labeling module configured to use the first sample word as a first sample label of the first sign language video, and use the combined text as a second sample label of the second sign language video; The training module is configured to train the sign language recognition model according to the first sign language video, the second sign language video, the first sample label and the second sample label until the sign language recognition model that meets the training stop condition is obtained.

13. A computing device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 10 or 11 are implemented.

14. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 10 or 11.

Images