Method, device and storage medium for screening least mouthful text

Abstract

The present disclosure provides a screening method, device and equipment of minimum mouth shape text and storage medium. First, a to-be-selected corpus and a predefined mouth shape bag are obtained, and then all similar syllable mouth shapes corresponding to each to-be-selected text are obtained respectively. Each to-be-selected text is traversed, and the syllable mouth shape text with the maximum number of similar syllable mouth shapes contained in the predefined mouth shape bag is selected from each to-be-selected text. When a syllable mouth shape text is selected each time, the pronunciation mouth shape corresponding to each similar syllable mouth shape of the syllable mouth shape text is removed from the predefined mouth shape bag, until the number of remaining pronunciation mouth shapes in the predefined mouth shape bag is less than a number threshold. Finally, the selected syllable mouth shape texts are used as the training data of the current mouth shape recognition. According to the coverage of all similar syllable mouth shapes of each to-be-selected text in the predefined mouth shape bag, the present disclosure screens the syllable mouth shape texts in descending order, and quickly screens the syllable mouth shape texts with the highest mouth shape coverage and the minimum number.

Description

Technical Field

[0001] This disclosure relates to the field of text filtering technology, and in particular to a method, apparatus, device, and storage medium for filtering text with minimal lip movements. Background Technology

[0002] When training lip-reading recognition on virtual avatars, the collection of training data is of great importance. If the minimum amount of training text can be collected, yet still cover all lip shapes, it can save a significant amount of data collection time and greatly improve the training speed of lip-reading recognition.

[0003] The current method for collecting training data for lip recognition is usually through brute-force retrieval, which has high algorithm complexity and takes a long time to filter training data. Summary of the Invention

[0004] The purpose of this disclosure is to provide a method, apparatus, device, and storage medium for filtering text with minimal lip movements, which can quickly filter syllable lip movement text with the most complete lip movements and the smallest data volume from a candidate corpus.

[0005] To achieve the above objectives, this disclosure adopts the following technical solution: a method for filtering text with minimal lip movements, comprising:

[0006] Obtain candidate corpus and predefined mouth shape word bag. The candidate corpus includes multiple candidate texts, and the predefined mouth shape word bag contains a preset number of pronunciation mouth shapes. The predefined mouth shape word bag represents the set of pronunciation mouth shapes that need to be collected in the current time.

[0007] Obtain the mouth shapes of all similar syllables corresponding to each of the candidate texts;

[0008] Iterate through each candidate text, and successively select the syllable mouth shape text with the largest number of similar syllable mouth shapes contained in the predefined mouth shape word bag from each candidate text. When a syllable mouth shape text is selected each time, remove the pronunciation mouth shapes corresponding to each of the similar syllable mouth shapes of the syllable mouth shape text from the predefined mouth shape word bag until the number of remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than the number threshold.

[0009] The selected syllable lip-sync texts are used as training data.

[0010] This disclosure also provides a filtering device for minimal lip-sync text, comprising:

[0011] The first acquisition module is used to acquire candidate corpus and predefined mouth shape word bag. The candidate corpus includes multiple candidate texts, and the predefined mouth shape word bag contains a preset number of pronunciation mouth shapes. The predefined mouth shape word bag represents the set of pronunciation mouth shapes that need to be collected in the current time.

[0012] The second acquisition module is used to acquire the mouth shapes of all similar syllables corresponding to each of the candidate texts;

[0013] The filtering module is used to traverse each candidate text, and successively filter out the syllable mouth shape text with the largest number of similar syllable mouth shapes contained in the predefined mouth shape word bag from each candidate text. When a syllable mouth shape text is selected each time, the pronunciation mouth shape corresponding to each of the similar syllable mouth shapes of the syllable mouth shape text is removed from the predefined mouth shape word bag until the number of remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than the number threshold.

[0014] The setting module is used to use the selected syllable lip-sync texts as training data.

[0015] This disclosure also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0016] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0017] This disclosure provides a method, apparatus, device, and storage medium for selecting texts with minimal lip movements. First, it acquires a candidate corpus and a predefined bag-of-mouth (BOM) for lip movements. The candidate corpus includes multiple candidate texts, and the BOM contains a preset number of lip movements, representing the set of lip movements to be collected in the current iteration. Then, it acquires all similar syllable lip movements corresponding to each candidate text. Next, it iterates through each candidate text, successively selecting the syllable lip movement text with the largest number of similar syllable lip movements contained in the BOM. Each time a syllable lip movement text is selected, the corresponding lip movements are removed from the BOM until the number of remaining lip movements in the BOM is less than a threshold. Finally, the selected syllable lip movement texts are used as training data for the current lip movement recognition. This disclosure filters texts by sorting them in descending order based on the coverage of all similar syllable mouth shapes in a predefined bag of mouth shapes for each candidate text, thereby quickly filtering out the syllable mouth shape texts with the highest mouth shape coverage and the fewest number of similar syllable mouth shapes. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the steps of a method for filtering text with minimal lip movements in one embodiment of this disclosure;

[0019] Figure 2 This is an overall structural block diagram of a screening device for minimal lip-sync text in one embodiment of this disclosure;

[0020] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present disclosure.

[0021] The realization of the purpose, functional features and advantages of this disclosure will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.

[0023] Reference Figure 1 One embodiment of this disclosure provides a method for filtering text with the fewest syllable lip movements, including:

[0024] S1: Obtain candidate corpus and predefined mouth shape word bag. The candidate corpus includes multiple candidate texts. The predefined mouth shape word bag contains a preset number of pronunciation mouth shapes. The predefined mouth shape word bag represents the set of pronunciation mouth shapes that need to be collected in the current time.

[0025] S2: Obtain the mouth shapes of all similar syllables corresponding to each of the candidate texts;

[0026] S3: Iterate through each candidate text, and successively select the syllable mouth shape text with the largest number of similar syllable mouth shapes contained in the predefined mouth shape word bag from each candidate text. When a syllable mouth shape text is selected each time, remove the pronunciation mouth shapes corresponding to each of the similar syllable mouth shapes of the syllable mouth shape text from the predefined mouth shape word bag until the number of remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than the number threshold.

[0027] S4: Use the selected syllable mouth shape texts as training data.

[0028] In this embodiment, the filtering system first acquires candidate corpora and a predefined bag-of-words (BOW). The candidate corpora include multiple candidate texts, and the predefined bag-of-words contains a preset number of pronunciation lip shapes, specifically defined as: r = k * w, where r represents the total number of pronunciation lip shapes contained in the predefined bag-of-words (i.e., the preset number), k is a hyperparameter representing k times the bag-of-words, and w is the number of commonly used syllables contained in the selected dictionary (preferably the *Modern Chinese Standard Dictionary* in this embodiment). The predefined bag-of-words represents the set of pronunciation lip shapes to be collected in this current iteration. That is, in this current iteration, the filtering system needs to sift out candidate texts from the candidate corpora that cover each pronunciation lip shape in the predefined bag-of-words. These candidate texts can subsequently be used as training data for lip shape recognition training.

[0029] The filtering system pre-constructs a confusion matrix and is based on the *Modern Chinese Standard Dictionary*, which contains w (specifically, w has a value of 410) different commonly used syllables. The system first converts each commonly used syllable in the dictionary into its corresponding common mouth shape code based on the pre-constructed syllable mouth shape encoding table. Then, it constructs a confusion matrix based on these common mouth shape codes. The confusion matrix has a size of w*w (i.e., 410*410), and the values ​​in the confusion matrix represent the similarity between two common mouth shape codes. The filtering system also uses the syllable mouth shape coding table to find the corresponding mouth shape codes for each candidate text in the candidate corpus. Then, it uses the confusion matrix to find all the similar syllable mouth shapes corresponding to the mouth shape codes of each candidate text. Then, it iterates through each candidate text and successively filters out the syllable mouth shape text with the largest number of similar syllable mouth shapes contained in the predefined mouth shape word bag. Each time a syllable mouth shape text is selected, the pronunciation mouth shapes corresponding to each similar syllable mouth shape text are removed from the predefined mouth shape word bag until the number of remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than the number threshold. Specifically, the filtering system iterates through each candidate text, and based on the candidate mouth shape code corresponding to each candidate text, it successively filters out several first candidate texts from each candidate text whose mouth shapes of similar syllables are included in a predefined mouth shape vocabulary. Each time a first candidate text is selected, the system removes the mouth shapes corresponding to the mouth shapes of the similar syllables of the first candidate text from the predefined mouth shape vocabulary (the purpose of this removal is to indicate that the mouth shapes in the predefined mouth shape vocabulary already have corresponding candidate texts, i.e., first candidate texts, and there is no need to repeatedly filter candidate texts with the same mouth shapes). After filtering out the first candidate texts from the candidate corpus whose mouth shapes of all similar syllables are included in the predefined mouth shape vocabulary, the filtering system determines whether the number of the first remaining mouth shapes in the predefined mouth shape vocabulary is less than a threshold. The first remaining mouth shapes are the mouth shapes in the predefined mouth shape vocabulary that are different from the mouth shapes of the similar syllables of each first candidate text (i.e., there are currently no candidate texts in the predefined mouth shape vocabulary that can correspond to their mouth shapes). If the number of remaining first-order pronunciation mouth shapes in the predefined bag of words is less than the threshold, no further text filtering is needed, and all first-selection texts can be used as training data for mouth shape recognition training. If the number of remaining first-order pronunciation mouth shapes in the predefined bag of words is greater than the threshold, then each second-selection text is iterated again, and several third-selection texts with the largest number of similar syllable mouth shapes are successively filtered from each second-selection text. Each time a second-selection text is selected, the pronunciation mouth shapes corresponding to the mouth shapes of each similar syllable of the second-selection text are removed from the predefined bag of words until the number of remaining second-order pronunciation mouth shapes in the predefined bag of words is less than the threshold.In this system, each second candidate text is a candidate text remaining after removing each first candidate text from the candidate corpus. The maximum number of similar syllable mouth shapes indicates that the second candidate text has the largest number of similar syllable mouth shapes included in the predefined bag of mouth shapes during a single selection. Finally, the selection system combines each first candidate text and each third candidate text to obtain the syllable mouth shape texts required for the current mouth shape recognition training as training data.

[0030] In this embodiment, the filtering system performs a step-by-step descending filtering based on the coverage rate of all similar syllable mouth shapes in the predefined mouth shape bag for each candidate text, thereby quickly filtering out the syllable mouth shape text with the highest mouth shape coverage rate and the fewest number of syllable mouth shapes.

[0031] Further, the step of traversing each candidate text, successively filtering out the syllable mouth shape text with the largest number of similar syllable mouth shapes contained in the predefined mouth shape word bag, and removing the pronunciation mouth shapes corresponding to each of the similar syllable mouth shapes of the selected syllable mouth shape text from the predefined mouth shape word bag each time a syllable mouth shape text is selected, until the number of remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than a number threshold, includes:

[0032] S301: Traverse each candidate text, and successively filter out a number of first candidate texts from each candidate text whose mouth shapes of all similar syllables are contained in the predefined mouth shape bag. When a first candidate text is selected each time, remove the pronunciation mouth shape corresponding to each of the mouth shapes of the similar syllables of the first candidate text from the predefined mouth shape bag.

[0033] S302: Determine whether the number of the first remaining pronunciation mouth shapes in the predefined mouth shape bag is less than the number threshold, wherein the first remaining pronunciation mouth shapes are the pronunciation mouth shapes in the predefined mouth shape bag that are different from the mouth shapes of the similar syllables in each of the first candidate texts;

[0034] S303: If the number of the first remaining pronunciation mouth shapes in the predefined mouth shape bag is greater than the number threshold, then each second candidate text is traversed again, and several third candidate texts with the largest number of similar syllable mouth shapes are selected from each second candidate text. When a second candidate text is selected each time, the pronunciation mouth shapes corresponding to the mouth shapes of each similar syllable of the second candidate text are removed from the predefined mouth shape bag until the number of the second remaining pronunciation mouth shapes in the predefined mouth shape bag is less than the number threshold. Each second candidate text is a candidate text remaining after removing each first candidate text from the candidate corpus. The largest number of similar syllable mouth shapes indicates that the second candidate text has the largest number of similar syllable mouth shapes in the predefined mouth shape bag during a single selection.

[0035] S304: Combine each of the first candidate texts and each of the third candidate texts to obtain each of the syllable mouth shape texts.

[0036] In this embodiment, based on the mouth shape encoding corresponding to the candidate text itself, the filtering system selects a first candidate text from each candidate text in a single instance from all candidate texts whose mouth shapes of all similar syllables are included in a predefined mouth shape word bag (i.e., when filtering candidate texts in a single instance, the filtering system first determines whether the mouth shapes of all similar syllables of the candidate text are included in the predefined mouth shape word bag; if the mouth shapes of all similar syllables of the candidate text are included in the predefined mouth shape word bag, then the candidate text is selected as the first candidate text; if the mouth shapes of all similar syllables of the candidate text are not included in the predefined mouth shape word bag, then the candidate text is skipped and the next candidate text in the candidate corpus is filtered). Each time a first candidate text is selected, the pronunciation mouth shapes corresponding to the mouth shapes of each similar syllable of the first candidate text are removed from the predefined mouth shape word bag (i.e., the predefined mouth shape word bag already contains the pronunciation mouth shapes of the corresponding candidate texts, so there is no need to filter syllable mouth shape texts as training data). The filtering system iterates through all candidate texts in the candidate corpus according to the above rules until all first candidate texts are selected. Then, it determines whether the number of first remaining mouth shapes in the predefined bag of mouth shapes is less than a threshold. The first remaining mouth shapes are the mouth shapes in the predefined bag of mouth shapes that are different from the mouth shapes of similar syllables in each first candidate text. If the number of first remaining mouth shapes in the predefined bag of mouth shapes is greater than the threshold, the system iterates through each second candidate text again, successively selecting several third candidate texts with the largest number of mouth shapes of similar syllables from each second candidate text. Each time a second candidate text is selected, the mouth shapes corresponding to the mouth shapes of each similar syllable in the second candidate text are removed from the predefined bag of mouth shapes until the number of second remaining mouth shapes in the predefined bag of mouth shapes is less than the threshold. Among them, each second candidate text is the candidate text remaining after removing each first candidate text from the candidate corpus. The maximum number of similar syllable mouth shapes indicates that the second candidate text has the largest number of displayed syllables included in the predefined bag of mouth shapes during a single screening.

[0037] For example, among all the second candidate texts, each candidate text contains n syllable mouth shapes, corresponding to m similar syllable mouth shapes. Of these m similar syllable mouth shapes, p syllable mouth shapes are included in a predefined bag-of-words for mouth shapes. For all the second candidate texts, p is less than m (for the first candidate text, p equals m). At this point, the filtering system iterates through each second candidate text, first selecting the third candidate text A with the largest p, and removing all mouth shapes corresponding to the similar syllable mouth shapes of the third candidate text A from the predefined bag-of-words for mouth shapes; then, it again selects the third candidate text B with the largest p, and removes all mouth shapes corresponding to the similar syllable mouth shapes of the third candidate text B from the predefined bag-of-words for mouth shapes; and so on, until the number of remaining second mouth shapes in the predefined bag-of-words for mouth shapes is less than a threshold.

[0038] Further, before the step of respectively obtaining all similar syllable mouth shapes corresponding to each of the candidate texts, it includes:

[0039] S5: Obtain a common dictionary, the common dictionary contains w different common syllables, and each of the common syllables has no tone;

[0040] S6: Respectively convert each of the common syllables into its corresponding common mouth shape code;

[0041] S7: Construct a confusion matrix according to each of the common mouth shape codes, the size of the confusion matrix is w*w, and the value in the confusion matrix represents the similarity between two of the common mouth shape codes.

[0042] In this embodiment, the screening system is based on the "Modern Chinese Standard Dictionary", and this common dictionary contains w different common syllables without tones. The screening system first converts each common syllable into its corresponding common mouth shape code through a pre-constructed syllable mouth shape code table, and then constructs a confusion matrix with a size of w*w according to all the common mouth shape codes of this dictionary. The value in this confusion matrix represents the similarity between two common mouth shape codes. Specifically, in the screening system, a pronunciation mouth shape code table is customized in advance according to the pronunciation methods of Chinese pinyin initials and finals. A syllable can find its corresponding digital code (i.e., mouth shape code) according to the pronunciation mouth shape code table. In the confusion matrix, the calculation steps for the similarity between two common mouth shape codes are as follows: respectively find the first common mouth shape code corresponding to the first common syllable and the second common mouth shape code corresponding to the second common syllable through the pronunciation code table (the pronunciation code is a string of numbers, for example, the syllable of "bang" is

bang

ba

[0043] Further, in the step of constructing a confusion matrix according to each of the common mouth shape codes, the calculation steps for the similarity between two of the common mouth shape codes include:

[0044] S701: Calculate the similarity between the first common mouth shape code and the second common mouth shape code according to the consistency of the values at each corresponding position between the first common mouth shape code and the second common mouth shape code.

[0045] In this embodiment, the developers pre-define a syllable mouth shape encoding table based on the initial consonant table and final vowel table in Mandarin Chinese Pinyin. The first encoding value can be determined according to the order of the syllable (or Pinyin) in the column of the initial consonant table, the second encoding value can be determined according to the order of the syllable in the row of the initial consonant table, the third encoding value can be determined according to the order of the syllable in the row of the final vowel table, and the fourth encoding value can be determined according to the order of the syllable in the column of the final vowel table.

[0046] The filtering system uses the predefined syllable mouth shape coding table to find the first commonly used mouth shape code for the first commonly used syllable and the second commonly used mouth shape code for the second commonly used syllable. Then, it calculates the similarity between the first and second commonly used mouth shape codes based on the consistency of their values ​​at each corresponding position. Specifically, the filtering system determines whether the values ​​at each position of the first and second commonly used mouth shape codes are consistent. If the values ​​at each position are consistent, the similarity is calculated based on the unit score and the total number of positions. If the values ​​at each position are partially the same, the system obtains the numerical sequence with the same value and calculates the similarity based on the unit score and the numerical sequence.

[0047] Furthermore, both the first commonly used lip shape code and the second commonly used lip shape code have four values. The step of calculating the similarity between the first commonly used lip shape code and the second commonly used lip shape code based on the consistency of the values ​​at each corresponding position includes:

[0048] S70101: Determine whether the values ​​at each position of the first commonly used lip shape code and the second commonly used lip shape code are consistent;

[0049] S70102: If the values ​​at each position of the first commonly used lip shape code and the second commonly used lip shape code are consistent, then the similarity between the first commonly used lip shape code and the second commonly used lip shape code is calculated based on the unit score and the total number of positions.

[0050] S70103: If the numerical portions at each position of the first commonly used lip shape code and the second commonly used lip shape code are the same, then obtain the numerical sequence with the same value, and calculate the similarity between the first commonly used lip shape code and the second commonly used lip shape code based on the unit score and each numerical sequence.

[0051] In this embodiment, both the first and second commonly used mouth shape codes have four values ​​(according to the embodiment corresponding to step S701 above, the four-digit code value of a syllable can be determined according to the pronunciation code table, that is, the pronunciation code corresponding to each pronunciation mouth shape is composed of four-digit code values). The screening system first judges whether the values ​​at each corresponding position of the first and second commonly used mouth shape codes are consistent (for example, comparing the first digit of the first commonly used mouth shape code with the first digit of the second commonly used mouth shape code, comparing the second digit of the first commonly used mouth shape code with the second digit of the second commonly used mouth shape code, and so on). If the values ​​at each position of the first and second commonly used mouth shape codes are consistent, the similarity between the first and second commonly used mouth shape codes is calculated by multiplying the unit score and the total number of positions. For example, if the unit score is 0.5 and the total number of positions is 4, the similarity obtained after multiplication is 2.0.

[0052] If the first and second commonly used mouth shape codes share only some numerical values, the specific similarity needs to be determined based on the order of the numerical values ​​(i.e., the order of the values ​​in the pronunciation code). Specifically, if the first, second, and third positions of the first and second commonly used mouth shape codes are the same, then only the number of positions with the same value needs to be multiplied by the unit score. For example, if the first and second commonly used mouth shape codes only share the same values ​​in the first and third positions, then the number of positions with the same value is 2, the unit score is 0.5, and therefore the similarity between the first and second commonly used mouth shape codes is 1.0. If the first and second commonly used mouth shape codes share the same value in the fourth position, then it is also necessary to check if their values ​​in the third position are the same. If the values ​​in the third position are the same, then the same value in the fourth position can obtain one unit score; if the values ​​in the third position are different, then the same value in the fourth position cannot obtain one unit score. The unit scores for other numerical positions are calculated according to the normal rules. That is, for other numerical positions, as long as the values ​​in that position are the same, that position can obtain a unit score. The sum of the unit scores obtained by all numerical positions is the similarity between the first and second most common lip shape codes.

[0053] Furthermore, in the step of obtaining the mouth shapes of all similar syllables corresponding to each of the candidate texts, the step of obtaining the mouth shapes of all similar syllables corresponding to a single candidate text includes:

[0054] S201: Obtain the candidate syllables of the candidate text;

[0055] S202: Look up the candidate mouth shape code corresponding to the candidate syllable according to the pre-constructed syllable mouth shape coding table;

[0056] S203: From the confusion matrix, look up all the common mouth shape codes whose similarity to the candidate mouth shape code is greater than the similarity threshold;

[0057] S204: Take each of the common mouth shape codes as all the similar syllable mouth shapes corresponding to the candidate text.

[0058] In this embodiment, the screening system first obtains the candidate syllables corresponding to a candidate text (simply understood as the pinyin of the candidate text, for example, the syllable of the candidate text "bang" is [bang]), and then looks up the mouth shape code corresponding to the candidate syllable according to the pre-constructed syllable mouth shape coding, that is, the candidate mouth shape code. Then, the screening system looks up all the common mouth shape codes in the confusion matrix whose similarity to the candidate mouth shape code is greater than the similarity threshold (for example, define the similarity threshold as 1.5, then all the common mouth shape codes whose similarity to the candidate mouth shape code is greater than 1.5 can be selected). Finally, the screening system takes each of the selected common mouth shape codes as all the similar syllable mouth shapes corresponding to the current candidate text.

[0059] Refer to Figure 2 , the present disclosure also provides a screening device for the text with the fewest syllable mouth shapes, including:

[0060] The first acquisition module 1 is used to acquire the candidate corpus and the predefined mouth shape word bag. The candidate corpus includes multiple candidate texts, and the predefined mouth shape word bag contains a preset number of pronunciation mouth shapes. The predefined mouth shape word bag represents the set of pronunciation mouth shapes that need to be collected in the current time;

[0061] The second acquisition module 2 is used to respectively acquire all the similar syllable mouth shapes corresponding to each of the candidate texts;

[0062] The screening module 3 is used to traverse each of the candidate texts, and successively screen out the syllable mouth shape text with the largest number of similar syllable mouth shapes included in the predefined mouth shape word bag from each of the candidate texts, and when each syllable mouth shape text is selected, remove the pronunciation mouth shapes corresponding to the similar syllable mouth shapes of the syllable mouth shape text from the predefined mouth shape word bag until the number of the remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than the quantity threshold;

[0063] The setting module 4 is used to take each of the selected syllable mouth shape texts as training data.

[0064] Further, the screening module 3 includes:

[0065] The first filtering unit is used to traverse each candidate text, and successively filter out a number of first candidate texts from each candidate text whose mouth shapes of all similar syllables are contained in the predefined mouth shape bag. When a first candidate text is selected each time, the pronunciation mouth shape corresponding to each of the mouth shapes of the similar syllables of the first candidate text is removed from the predefined mouth shape bag.

[0066] The judgment unit is used to determine whether the number of the first remaining pronunciation mouth shapes in the predefined mouth shape bag is less than the number threshold, wherein the first remaining pronunciation mouth shapes are the pronunciation mouth shapes in the predefined mouth shape bag that are different from the mouth shapes of the similar syllables in each of the first candidate texts;

[0067] The second filtering unit is configured to, if the number of the first remaining pronunciation mouth shapes in the predefined mouth shape bag is greater than the number threshold, then traverse each second candidate text again, and successively filter out a number of third candidate texts with the largest number of similar syllable mouth shapes from each second candidate text. Each time a second candidate text is selected, the pronunciation mouth shapes corresponding to the mouth shapes of each of the similar syllable mouth shapes of the second candidate text are removed from the predefined mouth shape bag until the number of the second remaining pronunciation mouth shapes in the predefined mouth shape bag is less than the number threshold. Each second candidate text is a candidate text remaining after removing each first candidate text from the candidate corpus. The largest number of similar syllable mouth shapes indicates that the second candidate text has the largest number of similar syllable mouth shapes included in the predefined mouth shape bag during a single filtering.

[0068] The synthesis unit is used to synthesize each of the first candidate texts and each of the third candidate texts to obtain each of the syllable mouth shape texts.

[0069] Furthermore, the screening device also includes:

[0070] The third acquisition module 5 is used to acquire a common dictionary, which contains w different common syllables, and each common syllable does not have a tone mark.

[0071] Conversion module 6 is used to convert each of the commonly used syllables into their respective commonly used lip shape codes;

[0072] Module 7 is used to construct a confusion matrix based on each of the commonly used lip-sync codes. The size of the confusion matrix is ​​w*w, and the values ​​in the confusion matrix represent the similarity between two commonly used lip-sync codes.

[0073] Furthermore, the building module 7 includes:

[0074] The calculation unit is used to calculate the similarity between the first commonly used lip shape code and the second commonly used lip shape code based on the consistency of the values ​​of the first commonly used lip shape code and the second commonly used lip shape code at each corresponding position.

[0075] Furthermore, both the first commonly used lip shape code and the second commonly used lip shape code have four values, and the calculation unit includes:

[0076] The judgment subunit is used to determine whether the values ​​at each position of the first commonly used lip shape code and the second commonly used lip shape code are consistent;

[0077] The first calculation subunit is used to calculate the similarity between the first commonly used lip shape code and the second commonly used lip shape code based on the unit score and the total number of positions if the values ​​at each position of the first commonly used lip shape code and the second commonly used lip shape code are consistent.

[0078] The second calculation subunit is used to obtain the numerical sequence with the same value if the numerical parts at each position of the first commonly used lip shape code and the second commonly used lip shape code are the same, and to calculate the similarity between the first commonly used lip shape code and the second commonly used lip shape code based on the unit score and each numerical sequence.

[0079] Furthermore, the second acquisition module 2 includes:

[0080] The acquisition unit is used to acquire the candidate syllables of the candidate text;

[0081] The first search unit is used to find the candidate lip shape code corresponding to the candidate syllable based on the pre-constructed syllable lip shape code table;

[0082] The second search unit is used to search the confusion matrix for all commonly used lip-sync codes that have a similarity greater than a similarity threshold with the candidate lip-sync code.

[0083] The setting unit is used to set each of the commonly used mouth shape codes as the mouth shapes of all the similar syllables corresponding to the candidate text.

[0084] In this embodiment, each module, unit, and subunit in the least syllable lip-sync text filtering device is used to perform each step in the least syllable lip-sync text filtering method described above, and the specific implementation process is not described in detail here.

[0085] This embodiment provides a minimum number of syllable lip-shape text filtering device. First, it acquires candidate texts and a predefined bag-of-mouth (BOM) for lip-shape recognition. The candidate texts include multiple candidate texts, and the BOM contains a preset number of lip-shapes, representing the set of lip-shapes to be collected in the current iteration. Then, it acquires all similar syllable lip-shapes corresponding to each candidate text. Next, it iterates through each candidate text, successively filtering out the syllable lip-shape text with the largest number of similar syllable lip-shapes contained in the BOM. Each time a syllable lip-shape text is selected, the lip-shapes corresponding to the similar syllable lip-shapes of that syllable lip-shape text are removed from the BOM until the number of remaining lip-shapes in the BOM is less than a threshold. Finally, the selected syllable lip-shape texts are used as training data for the current lip-shape recognition. This disclosure filters texts by sorting them in descending order based on the coverage of all similar syllable mouth shapes in a predefined bag of mouth shapes for each candidate text, thereby quickly filtering out the syllable mouth shape texts with the highest mouth shape coverage and the fewest number of similar syllable mouth shapes.

[0086] Reference Figure 3 This disclosure also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data such as candidate corpora. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for filtering text based on the fewest syllables of mouth movements.

[0087] The processor described above performs the following steps in the method for filtering text with the fewest syllables:

[0088] S1: Obtain candidate corpus and predefined mouth shape word bag. The candidate corpus includes multiple candidate texts. The predefined mouth shape word bag contains a preset number of pronunciation mouth shapes. The predefined mouth shape word bag represents the set of pronunciation mouth shapes that need to be collected in the current time.

[0089] S2: Obtain the mouth shapes of all similar syllables corresponding to each of the candidate texts;

[0090] S3: Iterate through each candidate text, and successively select the syllable mouth shape text with the largest number of similar syllable mouth shapes contained in the predefined mouth shape word bag from each candidate text. When a syllable mouth shape text is selected each time, remove the pronunciation mouth shapes corresponding to each of the similar syllable mouth shapes of the syllable mouth shape text from the predefined mouth shape word bag until the number of remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than the number threshold.

[0091] S4: Use the selected syllable mouth shape texts as training data.

[0092] Further, the step of traversing each candidate text, successively filtering out the syllable mouth shape text with the largest number of similar syllable mouth shapes contained in the predefined mouth shape word bag, and removing the pronunciation mouth shapes corresponding to each of the similar syllable mouth shapes of the selected syllable mouth shape text from the predefined mouth shape word bag each time a syllable mouth shape text is selected, until the number of remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than a number threshold, includes:

[0093] S301: Traverse each candidate text, and successively filter out a number of first candidate texts from each candidate text whose mouth shapes of all similar syllables are contained in the predefined mouth shape bag. When a first candidate text is selected each time, remove the pronunciation mouth shape corresponding to each of the mouth shapes of the similar syllables of the first candidate text from the predefined mouth shape bag.

[0094] S302: Determine whether the number of the first remaining pronunciation mouth shapes in the predefined mouth shape bag is less than the number threshold, wherein the first remaining pronunciation mouth shapes are the pronunciation mouth shapes in the predefined mouth shape bag that are different from the mouth shapes of the similar syllables in each of the first candidate texts;

[0095] S303: If the number of the first remaining pronunciation mouth shapes in the predefined mouth shape bag is greater than the number threshold, then each second candidate text is traversed again, and several third candidate texts with the largest number of similar syllable mouth shapes are selected from each second candidate text. When a second candidate text is selected each time, the pronunciation mouth shapes corresponding to the mouth shapes of each similar syllable of the second candidate text are removed from the predefined mouth shape bag until the number of the second remaining pronunciation mouth shapes in the predefined mouth shape bag is less than the number threshold. Each second candidate text is a candidate text remaining after removing each first candidate text from the candidate corpus. The largest number of similar syllable mouth shapes indicates that the second candidate text has the largest number of similar syllable mouth shapes in the predefined mouth shape bag during a single selection.

[0096] S304: Combine each of the first candidate texts and each of the third candidate texts to obtain each of the syllable mouth shape texts.

[0097] Furthermore, before the step of obtaining the mouth shapes of all similar syllables corresponding to each of the candidate texts, the following steps are included:

[0098] S5: Obtain a common dictionary, which contains w different common syllables, each of which does not have a tone mark;

[0099] S6: Convert each of the commonly used syllables into its corresponding commonly used mouth shape code;

[0100] S7: Construct a confusion matrix based on each of the commonly used lip-sync codes. The size of the confusion matrix is ​​w*w, and the values ​​in the confusion matrix represent the similarity between two commonly used lip-sync codes.

[0101] Furthermore, in the step of constructing a confusion matrix based on each of the commonly used lip-sync codes, the step of calculating the similarity between two commonly used lip-sync codes includes:

[0102] S701: Based on the consistency of the values ​​of the first commonly used lip shape code and the second commonly used lip shape code at each corresponding position, the similarity between the first commonly used lip shape code and the second commonly used lip shape code is calculated.

[0103] Furthermore, both the first commonly used lip shape code and the second commonly used lip shape code have four values. The step of calculating the similarity between the first commonly used lip shape code and the second commonly used lip shape code based on the consistency of the values ​​at each corresponding position includes:

[0104] S70101: Determine whether the values ​​at each position of the first commonly used lip shape code and the second commonly used lip shape code are consistent;

[0105] S70102: If the values ​​at each position of the first commonly used lip shape code and the second commonly used lip shape code are consistent, then the similarity between the first commonly used lip shape code and the second commonly used lip shape code is calculated based on the unit score and the total number of positions.

[0106] S70103: If the numerical portions at each position of the first commonly used lip shape code and the second commonly used lip shape code are the same, then obtain the numerical sequence with the same value, and calculate the similarity between the first commonly used lip shape code and the second commonly used lip shape code based on the unit score and each numerical sequence.

[0107] Furthermore, in the step of obtaining the mouth shapes of all similar syllables corresponding to each of the candidate texts, the step of obtaining the mouth shapes of all similar syllables corresponding to a single candidate text includes:

[0108] S201: Obtain the candidate syllables of the candidate text;

[0109] S202: Based on the pre-constructed syllable mouth shape coding table, find the mouth shape coding corresponding to the syllable to be selected;

[0110] S203: From the confusion matrix, find all commonly used lip-sync codes whose similarity to the candidate lip-sync codes is greater than a similarity threshold;

[0111] S204: Use the commonly used lip shape codes as the lip shapes of all similar syllables corresponding to the candidate text.

[0112] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for filtering text with the fewest syllable mouth shapes. The method for filtering text with the fewest syllable mouth shapes specifically includes:

[0113] S1: Obtain candidate corpus and predefined mouth shape word bag. The candidate corpus includes multiple candidate texts. The predefined mouth shape word bag contains a preset number of pronunciation mouth shapes. The predefined mouth shape word bag represents the set of pronunciation mouth shapes that need to be collected in the current time.

[0114] S2: Obtain the mouth shapes of all similar syllables corresponding to each of the candidate texts;

[0115] S3: Iterate through each candidate text, and successively select the syllable mouth shape text with the largest number of similar syllable mouth shapes contained in the predefined mouth shape word bag from each candidate text. When a syllable mouth shape text is selected each time, remove the pronunciation mouth shapes corresponding to each of the similar syllable mouth shapes of the syllable mouth shape text from the predefined mouth shape word bag until the number of remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than the number threshold.

[0116] S4: Use the selected syllable mouth shape texts as training data.

[0117] Further, the step of traversing each candidate text, successively filtering out the syllable mouth shape text with the largest number of similar syllable mouth shapes contained in the predefined mouth shape word bag, and removing the pronunciation mouth shapes corresponding to each of the similar syllable mouth shapes of the selected syllable mouth shape text from the predefined mouth shape word bag each time a syllable mouth shape text is selected, until the number of remaining pronunciation mouth shapes in the predefined mouth shape word bag is less than a number threshold, includes:

[0118] S301: Traverse each candidate text, and successively filter out a number of first candidate texts from each candidate text whose mouth shapes of all similar syllables are contained in the predefined mouth shape bag. When a first candidate text is selected each time, remove the pronunciation mouth shape corresponding to each of the mouth shapes of the similar syllables of the first candidate text from the predefined mouth shape bag.

[0119] S302: Determine whether the number of the first remaining pronunciation mouth shapes in the predefined mouth shape bag is less than the number threshold, wherein the first remaining pronunciation mouth shapes are the pronunciation mouth shapes in the predefined mouth shape bag that are different from the mouth shapes of the similar syllables in each of the first candidate texts;

[0120] S303: If the number of the first remaining pronunciation mouth shapes in the predefined mouth shape bag is greater than the number threshold, then each second candidate text is traversed again, and several third candidate texts with the largest number of similar syllable mouth shapes are selected from each second candidate text. When a second candidate text is selected each time, the pronunciation mouth shapes corresponding to the mouth shapes of each similar syllable of the second candidate text are removed from the predefined mouth shape bag until the number of the second remaining pronunciation mouth shapes in the predefined mouth shape bag is less than the number threshold. Each second candidate text is a candidate text remaining after removing each first candidate text from the candidate corpus. The largest number of similar syllable mouth shapes indicates that the second candidate text has the largest number of similar syllable mouth shapes in the predefined mouth shape bag during a single selection.

[0121] S304: Combine each of the first candidate texts and each of the third candidate texts to obtain each of the syllable mouth shape texts.

[0122] Furthermore, before the step of obtaining the mouth shapes of all similar syllables corresponding to each of the candidate texts, the following steps are included:

[0123] S5: Obtain a common dictionary, which contains w different common syllables, each of which does not have a tone mark;

[0124] S6: Convert each of the commonly used syllables into its corresponding commonly used mouth shape code;

[0125] S7: Construct a confusion matrix based on each of the commonly used lip-sync codes. The size of the confusion matrix is ​​w*w, and the values ​​in the confusion matrix represent the similarity between two commonly used lip-sync codes.

[0126] Furthermore, in the step of constructing a confusion matrix based on each of the commonly used lip-sync codes, the step of calculating the similarity between two commonly used lip-sync codes includes:

[0127] S701: Based on the consistency of the values ​​of the first commonly used lip shape code and the second commonly used lip shape code at each corresponding position, the similarity between the first commonly used lip shape code and the second commonly used lip shape code is calculated.

[0128] Furthermore, both the first commonly used lip shape code and the second commonly used lip shape code have four values. The step of calculating the similarity between the first commonly used lip shape code and the second commonly used lip shape code based on the consistency of the values ​​at each corresponding position includes:

[0129] S70101: Determine whether the values ​​at each position of the first commonly used lip shape code and the second commonly used lip shape code are consistent;

[0130] S70102: If the values ​​at each position of the first commonly used lip shape code and the second commonly used lip shape code are consistent, then the similarity between the first commonly used lip shape code and the second commonly used lip shape code is calculated based on the unit score and the total number of positions.

[0131] S70103: If the numerical portions at each position of the first commonly used lip shape code and the second commonly used lip shape code are the same, then obtain the numerical sequence with the same value, and calculate the similarity between the first commonly used lip shape code and the second commonly used lip shape code based on the unit score and each numerical sequence.

[0132] Furthermore, in the step of obtaining the mouth shapes of all similar syllables corresponding to each of the candidate texts, the step of obtaining the mouth shapes of all similar syllables corresponding to a single candidate text includes:

[0133] S201: Obtain the candidate syllables of the candidate text;

[0134] S202: Based on the pre-constructed syllable mouth shape coding table, find the mouth shape coding corresponding to the syllable to be selected;

[0135] S203: From the confusion matrix, find all commonly used lip-sync codes whose similarity to the candidate lip-sync codes is greater than a similarity threshold;

[0136] S204: Use the commonly used lip shape codes as the lip shapes of all similar syllables corresponding to the candidate text.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media provided in this disclosure and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0138] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, first object, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, first object, or method. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, first object, or method that includes that element.

[0139] The above description is only a preferred embodiment of this disclosure and does not limit the patent scope of this disclosure. Any equivalent structural or procedural changes made based on the content of this disclosure and its drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this disclosure.

Claims

1. A method of screening for minimal oral text, characterized by, The method comprises the following steps: obtaining candidate corpus and a pre-defined mouth shape bag, the candidate corpus comprising a plurality of candidate texts, the pre-defined mouth shape bag containing a preset number of pronunciation mouth shapes, and the pre-defined mouth shape bag representing a set of pronunciation mouth shapes to be collected in the current time; obtaining all similar syllable mouth shapes corresponding to each of the candidate texts respectively; traversing each of the candidate texts, and selecting, from each of the candidate texts, a syllable mouth shape text in which the number of similar syllable mouth shapes contained in the pre-defined mouth shape bag is the largest, and removing, from the pre-defined mouth shape bag, the pronunciation mouth shape corresponding to each of the similar syllable mouth shapes of the syllable mouth shape text each time one of the syllable mouth shape texts is selected, until the number of pronunciation mouth shapes remaining in the pre-defined mouth shape bag is less than a number threshold; using each of the selected syllable mouth shape texts as training data.

2. The method of claim 1, wherein, The step of traversing each of the candidate texts, and selecting, from each of the candidate texts, a syllable mouth shape text in which the number of similar syllable mouth shapes contained in the pre-defined mouth shape bag is the largest, and removing, from the pre-defined mouth shape bag, the pronunciation mouth shape corresponding to each of the similar syllable mouth shapes of the syllable mouth shape text each time one of the syllable mouth shape texts is selected, until the number of pronunciation mouth shapes remaining in the pre-defined mouth shape bag is less than a number threshold, comprises the following steps: traversing each of the candidate texts, and selecting, from each of the candidate texts, a plurality of first candidate texts in which all the similar syllable mouth shapes are contained in the pre-defined mouth shape bag, and removing, from the pre-defined mouth shape bag, the pronunciation mouth shape corresponding to each of the similar syllable mouth shapes of the first candidate text each time one of the first candidate texts is selected; determining whether the number of first remaining pronunciation mouth shapes of the pre-defined mouth shape bag is less than a number threshold, wherein the first remaining pronunciation mouth shapes are the pronunciation mouth shapes of the pre-defined mouth shape bag that are different from the similar syllable mouth shapes of each of the first candidate texts; if the number of first remaining pronunciation mouth shapes of the pre-defined mouth shape bag is greater than the number threshold, traversing each of second candidate texts again, and selecting, from each of the second candidate texts, a plurality of third candidate texts in which the number of similar syllable mouth shapes is the largest, and removing, from the pre-defined mouth shape bag, the pronunciation mouth shape corresponding to each of the similar syllable mouth shapes of the second candidate text each time one of the second candidate texts is selected, until the number of second remaining pronunciation mouth shapes of the pre-defined mouth shape bag is less than the number threshold, wherein each of the second candidate texts is a candidate text remaining in the candidate corpus after each of the first candidate texts is removed, and the largest number of similar syllable mouth shapes represents the largest number of similar syllable mouth shapes contained in the pre-defined mouth shape bag in a single selection; combing each of the first candidate texts and each of the third candidate texts to obtain each of the syllable mouth shape texts.

3. The method of claim 1, wherein the minimum mouthful text is filtered based on a number of characters in the minimum mouthful text. Before the step of obtaining all similar syllable mouth shapes corresponding to each of the candidate texts respectively, the method further comprises the following steps: Obtain a common dictionary, the common dictionary includes w different common syllables, each of the common syllables is not with tone; Each of the common syllables is converted into a corresponding common mouth shape code respectively; According to each of the common mouth shape code, a confusion matrix is constructed, the size of the confusion matrix is w*w, and the value in the confusion matrix represents the similarity between two common mouth shape codes.

4. The method of claim 3, wherein the minimum mouthful text is filtered based on a number of characters in the minimum mouthful text. In the step of constructing the confusion matrix according to each of the common mouth shape code, the calculation step of the similarity between two common mouth shape codes includes: According to the consistency of the numerical value of the first common mouth shape code and the second common mouth shape code at each corresponding position, the similarity between the first common mouth shape code and the second common mouth shape code is calculated.

5. The method of claim 3, wherein the minimum mouthful text is filtered by: In the step of obtaining all similar syllable mouth shapes corresponding to each of the candidate texts respectively, the step of obtaining all similar syllable mouth shapes corresponding to a single candidate text includes: Obtain the candidate syllable of the candidate text; According to the pre-constructed syllable mouth shape code table, the candidate mouth shape code corresponding to the candidate syllable is found; From the confusion matrix, all common mouth shape codes with a similarity greater than a similarity threshold value to the candidate mouth shape code are found; Each of the common mouth shape codes is used as all similar syllable mouth shapes corresponding to the candidate text.

6. A device for screening minimal mouthed text, characterized by, It includes: The first obtaining module is used for obtaining candidate corpus and pre-defined mouth shape bag, the candidate corpus includes a plurality of candidate texts, and the pre-defined mouth shape bag includes a preset number of pronunciation mouth shapes, the pre-defined mouth shape bag represents a set of pronunciation mouth shapes to be collected in the current time; The second obtaining module is used for obtaining all similar syllable mouth shapes corresponding to each of the candidate texts respectively; The screening module is used for traversing each of the candidate texts, and sequentially screening out the syllable mouth shape text with the maximum number of similar syllable mouth shapes contained in the pre-defined mouth shape bag from each of the candidate texts, and when one of the syllable mouth shape texts is selected, the pronunciation mouth shape corresponding to each of the similar syllable mouth shapes of the syllable mouth shape text is removed from the pre-defined mouth shape bag, until the number of remaining pronunciation mouth shapes in the pre-defined mouth shape bag is less than the number threshold value; The setting module is used for setting each of the selected syllable mouth shape texts as training data.

7. The device for screening minimal mouth text according to claim 6, characterized in that, The screening module includes: The first screening unit is used for traversing each of the candidate texts, and sequentially screening out a plurality of first candidate texts from each of the candidate texts, in which all similar syllable mouth shapes are contained in the pre-defined mouth shape bag, and when one of the first candidate texts is selected, the pronunciation mouth shape corresponding to each of the similar syllable mouth shapes of the first candidate text is removed from the pre-defined mouth shape bag; The judgment unit is used for judging whether the number of first remaining pronunciation mouth shapes in the pre-defined mouth shape bag is less than the number threshold value, wherein the first remaining pronunciation mouth shape is the pronunciation mouth shape different from the similar syllable mouth shape of each of the first candidate texts in the pre-defined mouth shape bag; a second screening unit, configured to: if the number of the first remaining pronunciation prons in the predefined pron bag is greater than the number threshold, traverse each second candidate text again, and successively screen a plurality of third candidate texts with the maximum number of similar syllable prons from each of the second candidate texts, and each time a second candidate text is selected, remove the pronunciation pron corresponding to each of the similar syllable prons of the second candidate text from the predefined pron bag until the number of the second remaining pronunciation prons in the predefined pron bag is less than the number threshold, wherein each of the second candidate texts is a candidate text remaining in the candidate corpus after each of the first candidate texts is removed, and the maximum number of similar syllable prons represents the maximum number of similar syllable prons of the second candidate text contained in the predefined pron bag in a single screening; a comprehensive unit, configured to: comprehensively combine each of the first candidate texts and each of the third candidate texts to obtain each of the syllable pron texts.

8. A computer device comprising a memory and a processor, the memory having stored therein a computer program, wherein, The processor executes the computer program to implement the steps of the method in any one of claims 1-5.

9. A computer readable storage medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the steps of the method in any one of claims 1-5.

Images