Language identification method and apparatus
By determining whether correction is needed from the output of the speech prediction model and optimizing language recognition using environmental features, the problem of inaccurate speech recognition in existing technologies is solved, achieving higher language recognition accuracy and personalized services.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SAMSUNG ELECTRONICS CHINA R&D CENT
- Filing Date
- 2022-11-22
- Publication Date
- 2026-07-24
AI Technical Summary
Existing speech-based language recognition technologies cannot provide accurate and reliable language recognition results, leading to inaccurate analysis of speech interaction content and an inability to provide users with reliable speech interaction services.
By comparing the output of the speech prediction model with a preset confidence difference threshold, it is determined whether correction is needed. The output is then optimized using language-related features in the current environment, or the language with the highest confidence is selected to improve the accuracy of language recognition.
It effectively improves the accuracy of language recognition based on speech, ensures the accuracy of speech interaction content analysis, and provides a better speech interaction experience and personalized services.
Smart Images
Figure CN115831096B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to artificial intelligence technology, and in particular to a language recognition method and apparatus. Background Technology
[0002] Language recognition involves identifying the language (i.e., the language) corresponding to the input text and speech. Currently, text-based language recognition is relatively simple and mature. However, speech-based language recognition technology is not yet mature. In commercial systems, the language is still manually set. This manual setting method cannot effectively adapt to situations where interactive languages vary.
[0003] Language recognition technology helps voice assistants parse users' voice interactions and provide further services based on the content. The accuracy of language recognition has a significant impact on speech recognition; accurate language recognition improves the system's ability to analyze multiple voice interactions, thus providing users with a better voice interaction experience. In particular, accurate language recognition results can serve as the foundation for public devices to provide services to users speaking different languages, and can also provide differentiated services based on the user's individual language.
[0004] In the process of developing this application, the inventors discovered that existing speech-based language recognition technologies cannot provide accurate and reliable language recognition results. Specifically, existing solutions mainly utilize pre-trained speech prediction models for language recognition. However, speech prediction models cannot guarantee high discriminative power in the output language analysis results. Therefore, it is impossible to make accurate language settings based on the output of the speech prediction model, thus failing to ensure accurate analysis of voice interaction content by the application system, and consequently, failing to provide users with reliable voice interaction services. Summary of the Invention
[0005] In view of this, the main objective of the present invention is to provide a language identification method and apparatus that can improve the accuracy of language identification based on speech.
[0006] To achieve the above objectives, the technical solution proposed in this embodiment of the invention is as follows:
[0007] A language identification method, comprising:
[0008] When a smart device needs to identify the language of a target object with which it interacts by voice, it uses a pre-trained language prediction model to predict the language based on the voice signal of the target object.
[0009] Based on the output of the language prediction model and the preset confidence difference threshold, determine whether the output of the language prediction model needs to be corrected.
[0010] If the correction is required, the output is optimized based on the language-related features in the current environment to obtain the language used by the target object; otherwise, the language with the highest confidence in the output is selected as the language used by the target object.
[0011] This invention also proposes a language identification device, comprising:
[0012] The speech prediction unit is used to predict the language based on the speech signal of the target object when the smart device needs to identify the language of the target object with which it interacts by voice.
[0013] The correction decision unit is used to determine whether the output of the language prediction model needs to be corrected based on the output of the language prediction model and a preset confidence difference threshold.
[0014] The correction unit is used to optimize the output result based on the language-related features in the current environment when the correction is required, so as to obtain the language used by the target object; when the correction is not required, the language with the highest confidence in the output result is selected as the language used by the target object.
[0015] This invention also provides an electronic device, including a processor and a memory;
[0016] The memory stores an application program that can be executed by the processor, which enables the processor to perform the language recognition method as described above.
[0017] This invention also proposes a computer-readable storage medium storing computer-readable instructions for performing the language recognition method described above.
[0018] In summary, the language identification scheme proposed in this embodiment of the invention determines whether the output of the speech-based language prediction model needs correction, and when correction is required, uses language-related features in the environment to correct the output. In this way, the impact of low discrimination of speech-based language identification results on the accuracy of language identification can be overcome, thereby effectively improving the accuracy of speech-based language identification. Attached Figure Description
[0019] Figure 1 This is a schematic diagram of the language identification method according to an embodiment of the present invention;
[0020] Figure 2 This is a schematic diagram illustrating the process of optimizing the model output based on language-related features in the current environment in an embodiment of the present invention.
[0021] Figure 3 This is a schematic diagram illustrating the process of determining the language used by a target object based on language-related features in the current environment and a modified set of language candidates in an embodiment of the present invention.
[0022] Figure 4 This is an example diagram illustrating the correction of model prediction results using an embodiment of the present invention;
[0023] Figures 5-8 This is an example diagram illustrating the application of the embodiments of the present invention in a specific scenario;
[0024] Figure 9 This is a schematic diagram of the language recognition device according to an embodiment of the present invention. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0026] Figure 1 This is a schematic diagram of the language recognition method according to an embodiment of the present invention, such as... Figure 1 As shown, the language identification method implemented in this embodiment mainly includes:
[0027] Step 101: When the smart device needs to identify the language of the target object with which it interacts by voice, it uses a pre-trained language prediction model to predict the language based on the voice signal of the target object.
[0028] This step involves inputting the speech signal of the interacting object into a pre-trained language prediction model when language identification is required. The model then predicts the language based on the speech signal and outputs a list of languages, including the confidence score for each language. This language identification can be implemented using existing speech-based language prediction models, which will not be elaborated upon here.
[0029] In practical applications, the timing for smart devices to perform language recognition on the target object with which they interact via voice can be set by those skilled in the art according to actual needs. For example, when the smart device detects that the target object has begun to interact with the device via voice, but it is not limited to this.
[0030] Step 102: Based on the output of the language prediction model and the preset confidence difference threshold, determine whether the output of the language prediction model needs to be corrected.
[0031] In this step, in order to address the issue that the language prediction model may output low-recognition language identification results, we will further determine whether it is necessary to correct the output results of the language prediction model. In order to improve the accuracy of language identification, when the language prediction model output results have low recognition, we can use language-related features in the current environment to correct the output results of the language prediction model.
[0032] The confidence difference threshold is used to determine whether the confidence levels of two languages are clearly distinguishable. When the confidence difference between the two languages is less than or equal to this threshold, it indicates that the distinction between the confidence levels of the two languages is small. Those skilled in the art can set an appropriate value for the confidence difference threshold based on the above principles and the actual application needs.
[0033] In one implementation, the need to correct the output can be determined based on the discriminant strength between the two languages with the highest confidence levels in the language prediction model's output. Specifically, this can be achieved using the following method:
[0034] If the confidence difference between the first and second languages in the first language sequence is not greater than the confidence difference threshold, then it is determined that the correction needs to be performed.
[0035] The first language sequence is the sequence obtained by sorting the languages in the output results in descending order of confidence.
[0036] Here, the first language in the first language sequence is the language with the highest confidence in the model output. If the confidence difference between the first and second languages in the first language sequence is not greater than a preset confidence difference threshold, it indicates that the distinction between these two languages is small. In this case, it cannot be guaranteed that the language with the highest confidence in the model output is the language used by the target object. Therefore, the aforementioned correction is required. Conversely, if the confidence difference between the first and second languages in the first language sequence is greater than the confidence difference threshold, it indicates that the distinction between these two languages is large. In this case, it can be definitively determined that the language used by the target object is the language with the highest confidence in the output. Therefore, no further correction to the output is required.
[0037] Step 103: If the correction is required, the output result is optimized based on the language-related features in the current environment to obtain the language used by the target object; otherwise, the language with the highest confidence in the output result is selected as the language used by the target object.
[0038] Here, when it is determined in step 102 that the output of the language prediction model needs to be corrected, the output will be optimized based on the language-related features in the current environment in order to select the language used by the target object from the output.
[0039] In one implementation, such as Figure 2 As shown, specifically, steps 201-202 can be used to optimize the output based on language-related features in the current environment to obtain the language used by the target object:
[0040] Step 201: Extract candidate sequences from the first language sequence, and obtain a modified language candidate set based on the candidate sequences; the first language sequence is a sequence obtained by sorting the languages in the output result in descending order of confidence; the candidate sequences satisfy the following: the confidence difference between adjacent languages in the sequence is less than or equal to the confidence difference threshold, and the first language of the sequence is the first language of the first language sequence.
[0041] This step is used to select the first few languages with lower distinguishability in the first language sequence from the output results, and construct a modified language candidate set so that in the subsequent step 202, the language used by the target object can be selected from the modified language candidate set based on the language-related features in the environment.
[0042] Step 202: Based on the language-related features in the current environment and the modified language candidate set, determine the language used by the target object.
[0043] In one implementation, such as Figure 3 As shown, specifically, steps 301-304 can be used to determine the language used by the target object based on the language-related features in the current environment and the modified language candidate set:
[0044] Step 301: Obtain the language-related features of each object in the current environment.
[0045] This step is used to collect language-related features of various objects in the environment.
[0046] In one embodiment, the language-related features can be set by those skilled in the art according to the actual application scenario, and may include, for example, text features, clothing features, human body features, voice signal features and / or location features, but are not limited thereto.
[0047] In practical applications, text features, clothing features, and human body features can be obtained based on the collected image data, and speech signal features can be obtained based on the collected environmental audio signals.
[0048] To facilitate the collection of language-related features of objects in the environment by intelligent devices, in practical applications, intelligent devices can monitor objects in the environment according to certain strategies, generating corresponding attribute data for each object. This accurately obtains the set of objects in the environment and the relevant data of each object. Then, when language identification is required, the language-related features of each object can be collected based on the current set of objects in the environment. Specifically, the following methods can be used to construct and maintain the object data in the environment:
[0049] The smart device detects objects in the environment according to a preset environmental object maintenance strategy and updates the object data of the current environment accordingly based on the detection results. The object data includes a set of objects and object attribute data; when the object is a person, the attribute data includes indications of whether it is a voice interaction object of the smart device and an object identifier; when the object is an item, the attribute data includes indications of whether the item is a personal item and indications of its owner.
[0050] The environmental object maintenance strategy can be specifically set by those skilled in the art according to actual needs. For example, it can be set to detect whether objects in the environment have changed based on a preset monitoring period or a preset event.
[0051] In practical applications, if long-term environmental monitoring is required, the following methods can be used to initialize objects in the environment:
[0052] For each target object in the environment, if the object is a person, then set the attribute data of the object and identify the items carried by the object, and generate attribute data for the identified items. In the attribute data, the items are set to be private items and the object to which they belong is the object. If the object is an item, then generate attribute data for the item. In the attribute data, the item is set to be a fixed item in the environment.
[0053] In practical applications, in order to improve the efficiency of collecting language-related features, those skilled in the art can select objects that usually have language-related features according to the actual application scenario and set the range of target objects to be identified.
[0054] When a target object interacting with the device is detected in the environment, the identification of the target object and its personal items is triggered, and the corresponding attribute data is set.
[0055] In order to obtain more language-related features that are beneficial to optimizing language recognition results, smart devices can collect speech signals in the environment in real time, identify the object to which the sound source belongs, and save the collected speech signals as speech data of the corresponding object, so that when language recognition is needed, speech signal features can be generated based on these speech data.
[0056] Step 302: According to the language relevance level of the objects, classify the language-related features of all the objects to obtain the language-related feature set corresponding to each language relevance level.
[0057] This step is used to classify all language-related features in the environment obtained in step 301 according to the language association level of the object, and obtain the language-related feature set corresponding to each language association level.
[0058] The language relevance level of an object is used to define the priority order for optimizing the language prediction output in step 101 based on the language-related features of different objects. The higher the language relevance level of an object, the more preferentially its language-related features are used to optimize the language prediction output.
[0059] In practical applications, those skilled in the art can set the language relevance level according to a preset language relevance level classification strategy.
[0060] In one implementation, the language relevance level classification strategy may specifically include rules:
[0061] If the object is an interactive object with the smart device or a personal item of the interactive object, then the language relevance level of the object is high.
[0062] If the object is someone other than the interactive object or someone else's personal property, then the language relevance level of the object is medium.
[0063] If the object is a fixed item in the environment, then the language relevance level of the object is low.
[0064] Step 303: For each language-related feature set, determine the feature-related language vector corresponding to each language-related feature in the language-related feature set, and generate a set-related language vector for the language-related feature set based on all the feature-related language vectors; wherein each element in the vector includes the language and the corresponding language correlation coefficient.
[0065] In one implementation, the feature-related language vector corresponding to each language-related feature in the language-related feature set can be determined using the following method:
[0066] For each language-related feature in the language-related feature set, if the language-related feature is a speech signal feature, then the language prediction model is used to predict the language. Based on the output of the language prediction model, a feature-related language vector corresponding to the language-related feature is generated. Otherwise, a preset mapping table between language-related features and associated languages is queried to obtain the feature-related language vector corresponding to the language-related feature.
[0067] In the above method, for non-speech signal features, the feature-related language vector corresponding to the language-related feature can be obtained by querying a preset mapping table of language-related features and associated languages.
[0068] The mapping table mentioned above can be specifically set by those skilled in the art based on experience, for example, as follows:
[0069] As shown in Table 1.
[0070] English text En_US / en_AU / en_GB w1 Non-interactor voice Determined by language model Model prediction results Miao ethnic costumes Zh_CN w2 Asian faces Zh_CN / ko_KR / ja_JP w3 / w4 / w5 Location: Mexico es-mx w6
[0071] Table 1
[0072] In Table 1 above, the language correlation coefficients can be set based on experience or obtained through data statistics. For example, if we statistically analyze the languages spoken by people of East Asian descent in videos on a certain video platform, assuming that 75% of them speak Chinese, 10% speak Japanese, 6% speak Korean, and 5% speak English, etc., we can set the values of w3, w4, and w5 in Table 1 accordingly.
[0073] In one implementation, the following method can be used to generate a set of language-related vectors for the language-related feature set based on all the aforementioned feature-related language vectors:
[0074] Add all the language-related vectors corresponding to the language-related feature set, and normalize the language correlation coefficient of each element in the addition result to obtain the set-related language vector of the language-related feature set.
[0075] In the above method, when adding the feature-related language vectors in the set, the language correlation coefficients of the same language are added together, and then the cumulative value of the language correlation coefficients of different languages is normalized.
[0076] Step 304: In descending order of language relevance level, traverse the language-related feature set sequentially. Based on the traversed language-related feature set, search for the language relevance coefficient corresponding to each language in the modified language candidate set from the language-related language vector of the set. If at least one of the searched language relevance coefficients is not zero, select the language corresponding to the largest language relevance coefficient from the search results as the language used by the target object, and end the traversal; otherwise, continue the traversal.
[0077] In this step, each language-related feature set is accessed in descending order of language relevance level. From the language vectors in each set, the language relevance coefficient corresponding to the language in the candidate language set is searched. If all the searched language relevance coefficients are zero, it means that the model output cannot be optimized based on the language vectors in the set corresponding to that language relevance level. At this time, it is necessary to continue accessing the language-related feature set corresponding to the next language relevance level until at least one language relevance coefficient is found that is not zero. Then, the language with the maximum value among the found language relevance coefficients is selected as the language used by the target object.
[0078] Assuming the language prediction model based on the speech of the interactive object outputs the prediction result as: M=[p1*L1,p2*L2,…..]p1>=p2>…; where p1 and p2 represent confidence levels, L1 and L2 represent languages, and the modified language candidate set consists of the top n languages with high confidence levels, i.e., the confidence levels of these languages are relatively close (the confidence difference between adjacent languages is less than or equal to the preset confidence difference threshold), belonging to the case where they cannot be clearly distinguished. The language vector related to each language-related feature set is as follows:
[0079] A = [a1*L1, a2*L2, ...]
[0080] B = [b1*L1, b2*L2, ...]
[0081] C = [c1*L1, b2*L2, ...]
[0082] The process iterates through the language vectors associated with each set, selecting the language correlation coefficients corresponding to the languages in the modified language candidate set. If a language in the modified language candidate set has a corresponding non-zero language correlation coefficient in the associated language vectors, the maximum value is selected as the language identification result. If a language in the modified language candidate set does not exist in the associated language vectors, meaning their language correlation coefficients are all zero, the same process is applied to B and C to obtain the final language.
[0083] Figure 4 An example of correcting model prediction results is given. As shown in the figure, using a pre-trained language prediction model, the first two languages with high confidence scores based on the speaker's speech are English and Spanish, with a confidence difference of 0.05. This indicates low discriminatory power between them, thus these two languages constitute a revised language candidate set. Next, language-related features based on the current environment are used to select the language used by the target object from this revised language candidate set. As shown in the figure, in a set of language-related vectors obtained based on language-related features in the current environment, the confidence scores for English and Spanish are 0.1 and 0.4 respectively. Therefore, the final language identification result can be determined as Spanish.
[0084] As can be seen from the above technical solution, after the language prediction model outputs the language prediction result, the language identification method in this application embodiment needs to further determine whether the output result of the language prediction model needs to be corrected. If correction is needed, the output result is corrected using language-related features in the environment. This overcomes the impact of low discriminative power of speech-based language identification results on the accuracy of language identification, thereby effectively improving the accuracy of speech-based language identification. The specific applications of the above solution are explained in detail below using several specific scenarios:
[0085] Application Example 1: Enhanced Language Recognition of Environmental Elements Based on Camera Vision. Smart devices can visually identify elements in the environment that represent a user's personal language preferences and enhance language recognition results based on the analysis of these elements.
[0086] Figure 5 and Figure 6 The following diagrams illustrate application example one. In this example, in a multilingual environment, the smart device can supplement the confidence level of a particular language in the language identification result by recognizing the linguistic inclinations of objects in the user's living environment, and ultimately assist the system in identifying the language used by the user.
[0087] like Figure 5 As shown in Application Example 1, when a user turns on the TV and asks in their native language, "Are there any good places to eat nearby?", the TV receives the voice command and analyzes the confidence level of the current sentence. The analysis results show that the three languages with high confidence (English, Spanish, and Persian) have very low distinguishability. Therefore, the camera is activated to capture relevant features in the environment. Since the captured document the user is viewing is in English, the user's language is ultimately determined to be English. Figure 6As shown in the first application example, after the TV identifies the language used by the user, it accurately analyzes the user's needs and recommends relevant food search results to the user.
[0088] Application Example 2: Language Recognition Enhancement Based on Environmental Sound Elements. Smart devices enhance the accuracy of language recognition for current user interaction speech by recognizing historical speech information in the environment.
[0089] Figure 7 A schematic diagram of application example two is provided. In this example, when a user communicates with a smart device using voice interaction in a multi-person voice scenario, the smart device can optimize the user's interaction language determination result by recognizing the language of the voice information related to the user in the environment.
[0090] like Figure 7 As shown, in a scenario where a student studying abroad invites a classmate to their home, user A asks a smart speaker: "What's the weather like tomorrow?" After detecting user A's voice command, the smart speaker performs language recognition based on the corresponding speech. The language recognition results show several languages with low discrimination: English, French, and German, with confidence levels of 30%, 32%, and 33%, respectively. Based on this, the smart speaker analyzes surrounding speech elements, detecting that user A previously conversed with users B and C, and analyzes the corresponding speech data to obtain the confidence level of the dialogue language: English 80%, French 30%, and German 55%. English has the highest confidence level; therefore, it is ultimately determined that user A used English. Based on this language recognition result, the smart speaker replies to user A in English: "Tomorrow's temperature is 45 degrees Fahrenheit, and the weather is sunny."
[0091] Application Example 3: Language Recognition Enhancement Based on User's Personal Items. In changing environments, smart devices can enhance the accuracy of language recognition for user-interacting speech by identifying elements on personal items surrounding the user. This allows for supplementary language recognition when the device struggles to identify the relevance of surrounding elements to the user in unpredictable environments, by recognizing elements on the user's personal items.
[0092] Figure 8 A schematic diagram of application example three is provided. For example... Figure 8As shown in the example, when a user interacts with a service robot in the mall, the service robot detects the user's voice command ("Where is the mall service desk?"). Based on the language recognition results, several languages with low discrimination are identified: English, French, and German, with confidence levels of 30%, 32%, and 33%, respectively. Based on this, the service robot uses its camera to detect that the language displayed on the lock screen of the user's personal device is English with a confidence level of 70%, and Arabic with a confidence level of 10%. Therefore, it ultimately determines that English is the language used by the user and responds in English, "Your destination is on the 3rd floor of the mall."
[0093] Based on the above method embodiments, correspondingly, the present invention also proposes a language identification device, such as... Figure 9 As shown, the device includes:
[0094] The speech prediction unit 901 is used to predict the language based on the speech signal of the target object when the smart device needs to identify the language of the target object with which it interacts by voice.
[0095] The correction decision unit 902 is used to determine whether the output of the language prediction model needs to be corrected based on the output of the language prediction model and a preset confidence difference threshold.
[0096] The correction unit 903 is used to optimize the output result based on the language-related features in the current environment when the correction is required, so as to obtain the language used by the target object; when the correction is not required, it selects the language with the highest confidence in the output result as the language used by the target object.
[0097] It should be noted that the above methods and apparatus are based on the same inventive concept. Since the methods and apparatus solve problems in similar ways, the implementation of the apparatus and methods can refer to each other, and the repeated parts will not be described again.
[0098] Based on the above method embodiments, this invention also proposes an electronic device, including a processor and a memory; the memory stores an application program executable by the processor, used to cause the processor to perform the language recognition method as described above. Specifically, a system or device equipped with a storage medium can be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or device can read and execute the program code stored in the storage medium. Furthermore, the operating system or other devices operating on the computer can perform some or all of the actual operations through instructions based on the program code. The program code read from the storage medium can also be written to a memory located in an expansion board inserted into the computer or to a memory located in an expansion unit connected to the computer, and then, based on the instructions of the program code, a CPU or other device installed on the expansion board or expansion unit can execute some or all of the actual operations, thereby realizing the functions of any of the above-described language recognition method embodiments.
[0099] Specifically, the memory can be implemented as various storage media such as electrically erasable programmable read-only memory (EEPROM), flash memory, and programmable programmable read-only memory (PROM). The processor can be implemented as one or more central processing units (CPUs) or one or more field-programmable gate arrays (FPGAs), wherein the FPGA integrates one or more CPU cores. Specifically, the CPU or CPU core can be implemented as a CPU or an MCU.
[0100] This application embodiment implements a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the language recognition method described above.
[0101] It should be noted that not all steps and modules in the above processes and structural diagrams are mandatory; some steps or modules can be omitted as needed. The execution order of the steps is not fixed and can be adjusted as required. The division of modules is merely for the convenience of description and functional division. In actual implementation, a module can be implemented by multiple modules, and the functions of multiple modules can also be implemented by the same module. These modules can be located in the same device or in different devices.
[0102] The hardware modules in each embodiment can be implemented mechanically or electronically. For example, a hardware module may include specially designed permanent circuitry or logic devices (such as dedicated processors, such as FPGAs or ASICs) to perform specific operations. A hardware module may also include programmable logic devices or circuitry (such as general-purpose processors or other programmable processors) temporarily configured by software to perform specific operations. The choice between mechanical implementation, dedicated permanent circuitry, or temporarily configured circuitry (such as software-configured circuitry) can be made based on cost and time considerations.
[0103] In this document, "illustrative" means "serving as an example, illustration, or description," and any illustration or embodiment described herein as "illustrative" should not be construed as a preferred or more advantageous technical solution. For the sake of brevity, the figures only schematically represent the parts relevant to the invention and do not represent their actual structure as a product. Furthermore, for the sake of clarity and ease of understanding, in some figures, components with the same structure or function are only schematically depicted, or only one is labeled. In this document, "a" does not mean that the number of relevant parts of the invention is limited to "only one," and "a" does not exclude the possibility that the number of relevant parts of the invention is "more than one." In this document, terms such as "upper," "lower," "front," "rear," "left," "right," "inner," and "outer" are used only to indicate the relative positional relationship between relevant parts, and not to limit the absolute position of these relevant parts.
[0104] The solutions described in this specification and embodiments, if involving the processing of personal information, will be processed only under the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be processed within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.
[0105] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A language identification method, characterized in that, include: When a smart device needs to identify the language of a target object with which it interacts by voice, it uses a pre-trained language prediction model to predict the language based on the voice signal of the target object. Based on the output of the language prediction model and the preset confidence difference threshold, determine whether the output of the language prediction model needs to be corrected. If the correction is required, the output is optimized based on the language-related features in the current environment to obtain the language used by the target object; otherwise, the language with the highest confidence in the output is selected as the language used by the target object. The optimization of the output based on language-related features in the current environment to obtain the languages used by the target object includes: Candidate sequences are extracted from the first language sequence, and a modified language candidate set is obtained based on the candidate sequences; Based on the language-related features in the current environment and the modified language candidate set, the language used by the target object is determined; The step of determining the language used by the target object based on language-related features in the current environment and the modified language candidate set includes: Obtain the language-related features of each object in the current environment; wherein, each object in the current environment is a person or an item; the target object is one of the people among the objects in the current environment; Based on the language relevance level of the objects, the language-related features of all the objects are classified to obtain the language-related feature set corresponding to each language relevance level; For each language-related feature set, determine the feature-related language vector corresponding to each language-related feature in the language-related feature set, and generate a set-related language vector for the language-related feature set based on all the feature-related language vectors; wherein, each element in the vector includes the language and the corresponding language correlation coefficient; The language-related feature sets are traversed sequentially in descending order of language relevance level. Based on the traversed language-related feature sets, the language relevance coefficients corresponding to each language in the modified language candidate set are searched from the language-related vectors of the set. If at least one of the searched language relevance coefficients is not zero, the language corresponding to the largest language relevance coefficient is selected from the searched language relevance coefficients as the language used by the target object, and the traversal ends; otherwise, the traversal continues.
2. The method according to claim 1, characterized in that, The step of determining whether the output of the language prediction model needs to be corrected includes: If the confidence difference between the first and second languages in the first language sequence is not greater than the confidence difference threshold, then it is determined that the correction needs to be performed; wherein, the first language sequence is the sequence after sorting the languages in the output result in descending order of confidence.
3. The method according to claim 1, characterized in that, The first language sequence is a sequence of languages in the output result sorted in descending order of confidence; the candidate sequence satisfies the following: the confidence difference between adjacent languages in the sequence is less than or equal to the confidence difference threshold, and the first language of the sequence is the first language of the first language sequence.
4. The method according to claim 1, characterized in that, The language relevance level is set according to a preset language relevance level classification strategy; the language relevance level classification strategy includes: If the object is an interactive object with the smart device or a personal item of the interactive object, then the language relevance level of the object is high. If the object is someone other than the interactive object or someone else's personal property, then the language relevance level of the object is medium. If the object is a fixed item in the environment, then the language relevance level of the object is low.
5. The method according to claim 1, characterized in that, The feature-related language vector corresponding to each language-related feature in the language-related feature set includes: For each language-related feature in the language-related feature set, if the language-related feature is a speech signal feature, then the language prediction model is used to predict the language. Based on the output of the language prediction model, a feature-related language vector corresponding to the language-related feature is generated. Otherwise, a preset mapping table between language-related features and associated languages is queried to obtain the feature-related language vector corresponding to the language-related feature.
6. The method according to claim 1, characterized in that, The process of generating a set of language-related feature vectors based on all the aforementioned feature-related language vectors includes: Add all the language-related vectors corresponding to the language-related feature set, and normalize the language correlation coefficient of each element in the addition result to obtain the set-related language vector of the language-related feature set.
7. The method according to claim 1, characterized in that, The method further includes: The smart device detects objects in the environment according to a preset environmental object maintenance strategy, and updates the object data of the current environment accordingly based on the detection results. The object data includes a set of objects and the attribute data of the objects. When the object is a person, the attribute data includes indication information as to whether it is a voice interaction object of the smart device and an object identifier. When the object is an item, the attribute data includes indication information as to whether the item is a private item and indication information as to which it belongs.
8. The method according to claim 1, characterized in that, The method further includes: The intelligent device collects voice signals from the environment in real time, identifies the object to which the sound source belongs, and saves the collected voice signals as the voice data of the corresponding object.
9. The method according to claim 1, characterized in that, The language-related features include text features, clothing features, human body features, speech signal features, and / or location features.
10. A language identification device, characterized in that, include: The speech prediction unit is used to predict the language based on the speech signal of the target object when the smart device needs to identify the language of the target object with which it interacts by voice. The correction decision unit is used to determine whether the output of the language prediction model needs to be corrected based on the output of the language prediction model and a preset confidence difference threshold. The correction unit is used to optimize the output result based on the language-related features in the current environment when the correction is required, so as to obtain the language used by the target object; when the correction is not required, the language with the highest confidence in the output result is selected as the language used by the target object. Specifically, the correction unit is used to extract candidate sequences from the first language sequence when obtaining the language used by the target object, and to obtain a corrected language candidate set based on the candidate sequences; and to determine the language used by the target object based on the language-related features in the current environment and the corrected language candidate set. Specifically, the correction unit is used to determine the language used by the target object based on the language-related features in the current environment and the correction language candidate set, by acquiring the language-related features of each object in the current environment; wherein each object in the current environment is a person or an object; the target object is one of the people among the objects in the current environment; classifying the language-related features of all objects according to the language relevance level of the objects to obtain a language-related feature set corresponding to each language relevance level; for each language-related feature set, determining the feature-related language vector corresponding to each language-related feature in the language-related feature set, and based on all the features... The relevant language vectors are generated by creating a set of relevant language vectors for the language-related feature set. Each element in the vector includes a language and a corresponding language correlation coefficient. The language-related feature set is traversed sequentially in descending order of language correlation level. Based on the traversed language-related feature set, the language correlation coefficients corresponding to each language in the modified language candidate set are searched from the set of relevant language vectors of the language-related feature set. If at least one of the searched language correlation coefficients is not zero, the language corresponding to the largest language correlation coefficient is selected from the searched language correlation coefficients as the language used by the target object, and the traversal ends; otherwise, the traversal continues.
11. An electronic device, characterized in that, Including processor and memory; The memory stores an application program that can be executed by the processor, which causes the processor to perform the language recognition method as described in any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that, It contains computer-readable instructions for performing the language identification method as described in any one of claims 1 to 9.