A language recognition method, device, equipment, storage medium and product

Abstract

The embodiment of the present application provides a language recognition method, device, equipment, storage medium and product. The technical scheme provided by the embodiment of the present application performs language recognition on the to-be-recognized voice information through a language recognition model, obtains a language recognition result of the to-be-recognized voice information, and the training voice sample used for training the language recognition model is obtained by collecting voice data of a target user with consistent recognition language labels and social language labels, so that the training voice sample used for further training of the language recognition model can be greatly expanded. The audio data of the target user with consistent recognition language and social language has higher language confidence in a multilingual scene, the reliability of the training voice sample is higher, the content of the training voice is more diverse, the recognition accuracy of the language recognition model obtained by training is higher in a complex language environment, and the language recognition accuracy in a complex multilingual scene is effectively improved.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to a language recognition method, apparatus, device, storage medium, and product. Background Technology

[0002] With the development of audio processing and internet technologies, users have placed higher demands on social platforms for personalized and accurate recommendations, as well as precise content recognition and understanding. For social platforms with multilingual user groups, the recognition, understanding, and recommendation of user content present even greater challenges. Complex multilingual scenarios require solving the language recognition problem first before better content recognition, understanding, and recommendation can be achieved. Therefore, language recognition plays an indispensable role in personalized and accurate language-based recommendations, and it is also the foundation for the implementation and performance improvement of other content recognition and understanding technologies.

[0003] For countries or regions with a relatively simple language, the language of the speech can be directly identified through IP address or country code. However, for countries or regions with a more complex language landscape, where there may be a large number of users speaking different languages ​​within the same country or region, it is difficult to determine the accurate language of the speech through IP address or country code, resulting in low language recognition accuracy. Summary of the Invention

[0004] This application provides a language recognition method, apparatus, device, storage medium, and product to solve the technical problem of low language recognition accuracy in complex multilingual scenarios, and effectively improve the language recognition accuracy in complex multilingual scenarios.

[0005] In a first aspect, embodiments of this application provide a language identification method, including:

[0006] Obtain the voice information of the user to be identified;

[0007] The speech information to be identified is input into the trained language recognition model, and the language recognition model is used to identify the language of the speech information to be identified, so as to obtain the language recognition result corresponding to the user to be identified. The language recognition model is trained by pre-collected training speech samples, which are obtained by collecting speech data from target users whose recognition language labels and social language labels are consistent.

[0008] In a second aspect, embodiments of this application provide a language recognition device, including a voice acquisition module and a language recognition module, wherein:

[0009] The voice acquisition module is configured to acquire the voice information of the user to be identified;

[0010] The language recognition module is configured to input the speech information to be recognized into a trained language recognition model, and perform language recognition on the speech information to be recognized through the language recognition model to obtain the language recognition result corresponding to the user to be recognized. The language recognition model is trained by pre-collected training speech samples, which are obtained by collecting speech data from target users whose recognition language tags and social language tags are consistent.

[0011] In a third aspect, embodiments of this application provide a language recognition device, including: a memory and one or more processors;

[0012] The memory is used to store one or more programs;

[0013] When the one or more programs are executed by the one or more processors, the one or more processors implement the language recognition method as described in the first aspect.

[0014] In a fourth aspect, embodiments of this application provide a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the language recognition method as described in the first aspect.

[0015] In a fifth aspect, embodiments of this application provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from the computer-readable storage medium and executes the computer program, causing the device to perform the language recognition method as described in the first aspect.

[0016] This application embodiment uses a language recognition model to identify the language of the speech information to be identified, obtaining the language recognition result of the speech information to be identified. The training speech samples used to train the language recognition model are obtained by collecting speech data from target users whose identification language labels and social language labels are consistent. The training speech samples can be further expanded on a large scale. The audio data of target users whose identification language and social language are consistent have higher language confidence in multilingual scenarios, higher reliability of training speech samples, more diverse content of training speech, and higher recognition accuracy of the trained language recognition model in complex language environments, effectively improving the language recognition accuracy in complex multilingual scenarios. Attached Figure Description

[0017] Figure 1 This is a flowchart of a language identification method provided in an embodiment of this application;

[0018] Figure 2This is a schematic diagram of a training speech sample collection process provided in an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of a language tag determination process provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of a social language tag determination process provided in an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of the structure of a language recognition device provided in an embodiment of this application;

[0022] Figure 6 This is a schematic diagram of the structure of a language recognition device provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but additional steps not included in the drawings may also be present. The above processes can correspond to methods, functions, procedures, subroutines, subroutines, etc.

[0024] The language recognition method provided in this application can be applied to language recognition scenarios, such as recognizing the language of user-provided speech information. Based on the language recognition results, it can be used for recommendation delivery, content review, content recognition and understanding, etc. The aim is to train a language recognition model using large-scale training speech samples obtained from collecting speech data from target users whose language tags and social language tags match. This trained model can then achieve accurate language recognition in complex multilingual scenarios. Traditional language recognition schemes typically rely on the IP address or country code of electronic devices for confirmation. However, for countries or regions with complex language types, it is difficult to determine the accurate language of speech solely through IP address or country code. To improve language recognition accuracy, language recognition models (such as deep learning models) can be trained. However, each mature language system has a vast vocabulary and complex grammar, and there are significant differences in vocabulary and grammar between different languages. Even within the same language, people from different regions have different accents and dialects. Furthermore, in real-world social chat scenarios, various environmental factors can interfere, making accurate language recognition difficult in complex multilingual environments. A large amount of audio training data is needed to train a high-precision language recognition model. Currently, most publicly available language datasets come from television programs or formal speeches, which differ significantly from chat scenarios. Public datasets have low confidence levels in complex language environments, and directly using them for model training can easily lead to poor performance of language recognition models in real-world social chat scenarios. Alternatively, audio can be collected and labeled directly from business scenarios, but the high cost of labeling limits the amount of labeled data and thus the accuracy of the language recognition model. Based on this, an embodiment of the present application provides a language recognition method to solve the technical problems of low language recognition accuracy, low confidence of model training data, and low work efficiency of a large number of audio acquisition and annotation operations in complex multilingual scenarios of existing language recognition schemes.

[0025] Figure 1 A flowchart of a language recognition method provided in an embodiment of this application is given. The language recognition method provided in this embodiment of the application can be executed by a language recognition device, which can be implemented by hardware and / or software and integrated into a language recognition device (such as a server, cloud platform, etc.).

[0026] The following description uses a language recognition device to perform a language recognition method as an example. (Reference) Figure 1 The language identification method includes:

[0027] S110: Obtain the voice information of the user to be identified.

[0028] For example, the solution obtains the voice information to be processed provided by the user to be identified, which requires language recognition. The voice information to be processed provided by this solution can be audio recorded by the user through an electronic device (such as a mobile phone, tablet, etc.) using an audio acquisition module (such as the microphone of the electronic device), or it can be obtained by the user from the local storage location of the electronic device, or from a download address specified by the user.

[0029] The users to be identified in this solution can be understood as those requiring language identification to determine language tags. Optionally, language identification can be performed in response to a user's language identification request, or it can be based on the need for language identification. For example, when it is necessary to perform accurate recommendations, content review, content recognition and understanding based on the user's language tags, the user's voice information to be identified can be obtained for language identification, and corresponding subsequent processing can be performed based on the language tags corresponding to the language identification.

[0030] In one possible embodiment, the speech information to be recognized can be actively provided by the user, such as by the user recording the speech information using an audio acquisition module and uploading it to a language recognition device, or it can be obtained from the audio data of the content to be recognized provided by the user. Alternatively, the speech information can be obtained from pre-collected audio data of the user, such as audio data stored by the user on a cloud server, where the audio data stored on the cloud server can be audio data recorded during interactions between the user and other users.

[0031] S120: Input the speech information to be recognized into the trained language recognition model, and use the language recognition model to recognize the language of the speech information to be recognized, and obtain the language recognition result corresponding to the user to be recognized. The language recognition model is trained by pre-collected training speech samples. The training speech samples are obtained by collecting speech data from target users whose recognition language labels and social language labels are consistent.

[0032] The language recognition device provided in this solution is pre-configured with a pre-trained language recognition model. This model is trained based on pre-collected training speech samples, which are obtained by collecting speech data from target users whose language tags match the recognition language tags and social language tags. Optionally, the language recognition model provided in this solution can be built based on classification models such as Conformer, Transformer, and ECAPA-TDNN.

[0033] For example, for candidate users (i.e., users used as training speech samples), the language recognition label and social language label for each candidate user are determined. Candidate users whose language recognition label and social language label match are identified as target users, and audio data is collected based on the identified target users. After collecting training speech samples corresponding to multiple target users, the pre-built language recognition model can be trained using the training speech samples to obtain the trained language recognition model.

[0034] Optionally, when training the language recognition model, the language recognition model can be trained using training speech samples or the speech features of training speech samples as output, and the sample speech labels of training speech samples as output. In one embodiment, the sample speech labels of training speech samples can be the recognition language label or social language label corresponding to the target user.

[0035] It should be explained that the language identification tag provided in this solution can be understood as a language tag used to identify and confirm the user's language, while the social language tag can be understood as a language tag determined based on the user's social relationships. It is understood that for users whose identification language tag and social language tag are consistent, it can be assumed that the language used by the user is highly consistent with the language used in their social activities, and these users have a high degree of confidence in their language use in complex language environments.

[0036] For example, after acquiring the speech information of the user to be identified, the speech information is input into a trained language recognition model. Upon receiving the speech information, the language recognition model performs language recognition processing to obtain and output the language label corresponding to the speech information. This language label is the language recognition result for the user to be identified. Optionally, the language recognition result can be represented in the form of a language label.

[0037] In one possible embodiment, after obtaining the language identification result for the user to be identified, response processing can be performed based on the language identification result. Optionally, the response processing based on the language identification result can include recommending and distributing content to the user to be identified, content review, and content recognition and understanding. For example, when recommending and distributing content to the user to be identified, after determining the language identification result, content in the corresponding language (such as voice chat rooms, short videos, etc.) can be accurately recommended or distributed to the user on the social platform, effectively improving user engagement time and retention. It is understandable that on multilingual social platforms, content review in different languages ​​is inevitable, but reviewers usually only have proficiency in one language. In order for reviewers of the corresponding language to better understand the content to be reviewed, the platform needs to classify the content to be reviewed by language and assign it to the review team of the corresponding language. This solution, after determining the language identification result of the user to be identified, classifies the content to be reviewed by language based on the language identification result and pushes the content to be reviewed to the review team of the corresponding language, effectively improving the review efficiency of the reviewers and further ensuring the content security of the platform. Furthermore, in cross-language scenarios, due to the significant differences between different languages, content recognition and understanding models cannot directly use a general model to solve content understanding in different languages. However, this solution outputs the language recognition result of the user to be identified through the language recognition model, and then calls the corresponding language speech recognition model to perform content recognition and understanding, effectively reducing the difficulty of content recognition and understanding tasks and improving the accuracy of recognition and understanding models.

[0038] In one possible embodiment, such as Figure 2 A schematic diagram of a training speech sample collection process is provided. The training speech sample collection steps provided by this scheme include S101-S104:

[0039] S101: Obtain initial speech samples from multiple sample users and determine the language label for each sample user based on the initial speech samples.

[0040] For example, multiple candidate sample users are identified, and initial voice samples from these users are obtained. The sample users provided in this solution can be understood as candidate users waiting to be selected as target users. It should be explained that some sample users provided in this solution have inconsistent language labels between the recognition language and the social language label. The language confidence of these sample users is low. If the voice data of these sample users is added to the training of the language recognition model, it can easily affect the accuracy of the language recognition model in complex language environments. This solution needs to filter out sample users with low language confidence and retain sample users with high language confidence as target users, and then use the audio data of the target users to train the language recognition model.

[0041] In one embodiment, the initial sample audio provided by this solution can be obtained by the sample user through an electronic device using an audio acquisition module, or it can be obtained by the sample user from the local storage location of the electronic device, or from a download address specified by the sample user, or it can be obtained by pulling the audio data cached by each sample user from the cloud.

[0042] After obtaining initial speech samples from multiple users, the language label for each user can be determined based on those initial speech samples. For example, language recognition can be performed based on the initial speech samples, or the language can be determined based on the user or electronic device, thereby determining the user's language label.

[0043] In one possible embodiment, such as Figure 3 The provided schematic diagram illustrates a language label determination process. This solution, based on initial speech samples, determines the language label for each user sample, including:

[0044] S1011: Based on the initial speech samples, determine one or more combinations of device language labels, user language labels, and region language labels for each sample user. The user language label is obtained by performing language recognition on the initial speech samples based on the language recognition model to be trained.

[0045] S1012: Determine the identification language label for each sample user based on one or more combinations of device language label, user language label, and regional language label.

[0046] For example, the language label provided by this solution can be determined based on one or more combinations of the device language label, user language label, and regional language label of the sample user. After obtaining initial voice samples from multiple sample users, one or more combinations of the device language label, user language label, and regional language label of each sample user are determined based on the initial voice samples.

[0047] Furthermore, based on one or more combinations of device language tags, user language tags, and regional language tags, the identification language tag for each sample user is determined. Optionally, taking the determination of the identification language tag based on a combination of device language tags, user language tags, and regional language tags as an example, after determining the device language tags, user language tags, and regional language tags, the language tag with the highest proportion is used as the identification language tag. When the device language tags, user language tags, and regional language tags are all different, the identification language tag can be determined according to the priority of different language tag types (e.g., the priority of device language tags, user language tags, and regional language tags decreases in that order). This solution improves the accuracy of target user screening and the recognition accuracy of the language recognition model by accurately determining the identification language tag of sample users based on device language tags, user language tags, and regional language tags.

[0048] In one possible embodiment, this scheme determines the identification language label of each sample user based on one or more combinations of device language labels, user language labels, and regional language labels. This can be done by determining the identification language label of each sample user from one or more combinations of device language labels, user language labels, and regional language labels according to a set label type priority.

[0049] For example, this solution sets the priority of the label type in descending order for device language label, user language label and regional language label. That is, the sample user is labeled according to the priority order of device language label, user language label and regional language label to determine the identification language label of the sample user.

[0050] The user language tags provided in this solution can be obtained by using a language recognition model to identify the language of initial speech samples. For example, the language recognition model identifies the language of the initial speech samples, determines the scores of the initial speech samples under different language tags, and uses the language tag with the highest score as the recognition result. This solution determines the language tags based on the priority of different tag types, improving the accuracy of target user screening and the recognition accuracy of the language recognition model.

[0051] Optionally, the language recognition model to be trained provided in this solution can be an initial language recognition model or a language recognition model obtained from the previous training during the iterative upgrade process of the language recognition model. The initial language recognition model can be trained based on a dataset with pre-annotated language labels. For example, it can obtain a language dataset from manually labeled business scenarios (e.g., chat scenarios) (the data volume of the language dataset can be a small-scale dataset, such as collecting 15 hours of voice data for each language), and use this language dataset to train a language recognition model based on a neural network to obtain the initial language recognition model.

[0052] Optionally, the initial language recognition model provided by this solution can be trained based on pre-collected training speech samples through small-sample annotation and / or training speech data pre-collected from a single language region. For example, for training speech data obtained through small-sample annotation, a small-scale language dataset is collected, and the language label for each training speech data in the language dataset is manually labeled. For training speech data obtained from a single language region, a language dataset is collected in that region, and the language labels for the training speech data in the language dataset are uniformly labeled based on the language used in that region. This solution can train the initial language recognition model using training speech data obtained through small-sample annotation and / or training speech data collected from a single language region, effectively improving the convergence speed of the language recognition model under multilingual training samples and increasing model training efficiency.

[0053] The regional language label provided by this solution can be determined based on the default language of the sample user's region. For example, it can be obtained by obtaining the IP address or country code of the electronic device used by the sample user, and then determining the regional language label based on the region or country language mapped by the IP address or country code.

[0054] The device language tag provided by this solution can be determined based on the language used by the electronic devices carried by the sample users. For example, if a sample user sets the default language of their electronic device based on their own language habits, the device language tag can be determined based on this default language. This solution accurately determines the identification language tag of sample users by combining prior knowledge of language recognition (the initial language recognition model) and user information (regional language and device language), thereby improving the accuracy of target user screening and the recognition accuracy of the language recognition model.

[0055] S102: Obtain the social relationship information of each sample user, and determine the social language tag of each sample user based on the social relationship information.

[0056] For example, the social relationship information of each sample user is obtained, and the corresponding social language tag for each sample user is determined based on the social relationship information. The social relationship information provided by this solution can be used to indicate the social interactions between sample users and other related users (such as the sample user's friends), as well as the language used in these interactions. It should be explained that assuming there is a high frequency of social interaction between two users, it can be understood that there is no language barrier between the two users, and the language used by the two users is highly consistent. Therefore, the corresponding social language tag can be accurately determined based on the language used in the social interactions between the sample user and each related user.

[0057] In one possible embodiment, such as Figure 4As shown in the schematic diagram of a social language tag determination process, the language identification method provided in this solution, when determining the social language tags of each sample user based on social relationship information, includes:

[0058] S1021: Determine the social relationship user circle corresponding to each sample user based on social relationship information. The social relationship user circle records the associated users and associated languages ​​of the sample user.

[0059] S1022: Determine the social language tag for each sample user based on the number of associated users in different related languages ​​within the social relationship user circle.

[0060] For example, after obtaining the social relationship information of sample users, the social relationship user circle corresponding to each sample user is determined based on the corresponding social relationship information. The social relationship user circle provided by this solution records associated users who have social interactions with the sample users and the associated languages ​​used in their social interactions.

[0061] For example, based on social relationship information, associated users with whom the sample user has social interactions are identified, and the language used by the sample user when interacting with each associated user is determined. This language is the associated language between the sample user and the corresponding associated user. Based on each associated user and their corresponding associated language, the sample user's social relationship user circle is determined. Optionally, the associated language can be determined based on the languages ​​used by the sample user and associated users. For example, the languages ​​used by the sample user and associated users during social interactions can be identified separately, and if the languages ​​used by both are consistent, that language is identified as the associated language.

[0062] In one possible embodiment, the associated users in the social relationship user circle provided by this solution can be determined based on one or more combinations of the sample user's audio / video call relationships, live interaction relationships, message chat relationships, and address book relationships. For example, audio / video call records (e.g., voice / video calls between users), live interaction records (e.g., live streamers and viewers, or hosts connecting with each other), message chat records (e.g., chats based on SMS or social software), and address books of the sample user can be obtained from social platforms and user devices to determine the sample user's audio / video call relationships, live interaction relationships, message chat relationships, address book relationships, and other social interaction relationships, and the social relationship user circle corresponding to the sample user can be determined based on the above social interaction relationships. This solution accurately determines the associated users of the sample user in the social relationship user circle based on the sample user's audio / video call relationships, live interaction relationships, message chat relationships, and address book relationships, improving the efficiency and accuracy of social language tag determination and effectively improving the accuracy of target user screening.

[0063] In one embodiment, after determining the social relationship user circles corresponding to each sample user, the languages ​​existing in these user circles are identified, and these languages ​​are designated as the associated languages ​​of the corresponding sample users. The number of associated users for each of these associated languages ​​is also determined. Furthermore, the associated language with the largest number of associated users in the social relationship user circle is used as the social language tag for the sample user. This solution determines the social language tag based on the number of associated users for different associated languages ​​in the social relationship user circle, defines a batch of social relationship user circles for each sample user, and then uses the language with the largest number of associated users in each social relationship user circle as the new user relationship rendering language (i.e., the social language). This improves the efficiency and accuracy of determining the language tag for sample users and enhances the accuracy of target user selection.

[0064] S103: Based on the identification language tags and social language tags, the sample users are filtered to obtain multiple target users.

[0065] For example, after determining the language identification tags and social language tags of multiple sample users, the sample users are filtered based on their respective language identification tags and social language tags to obtain multiple target users. This solution effectively improves the language identification accuracy of the language identification model in complex language environments by filtering the sample users based on their language identification tags and social language tags to obtain target users with higher language confidence.

[0066] In one possible embodiment, when filtering sample users based on identified language tags and social language tags to obtain multiple target users, this solution can select sample users whose identified language tags and social language tags are consistent as target users. For example, the identified language tag and social language tag corresponding to each sample user are compared, and sample users with the same identified language tag and social language tag are selected as target users. This solution obtains target users by filtering sample users based on identified language tags and social language tags, ensuring consistency between the identified language and social language of the target users. Even in complex language environments, the language used by the target user can still be accurately identified. Training speech samples collected from the target users can train a language recognition model with higher recognition accuracy, improving language recognition accuracy in complex language environments.

[0067] S104: Collect voice data from each target user to obtain training voice samples.

[0068] For example, voice data is collected from each of the aforementioned target users, and the collected voice data is used as training voice samples. After collecting the training voice samples, the language recognition model to be trained can be trained using the training voice samples, and the trained language recognition model can be configured into the language recognition device. Optionally, when training the language recognition model using the training voice samples, the training voice samples or their audio features can be used as input, and the language labels of the training voice samples can be used as output. The language labels of the samples are consistent with the target user's recognition language label or social language label. Optionally, after collecting the training voice samples corresponding to the target users, the training voice samples can be automatically labeled based on the target user's recognition language label or social language label, eliminating the need for manual labeling of the training voice samples and improving the efficiency of labeling and collecting training voice samples.

[0069] This solution collects voice data from target users whose language tags match the social language tags. This collected voice data is then used as training samples for the language recognition model. This improves the language confidence of the training voice samples, increases the amount of training voice sample data at a lower cost, and simultaneously improves the efficiency of training voice sample collection. Furthermore, by expanding the amount of training voice sample data, a better language recognition model can be iteratively developed based on the training voice samples. Because of the large expansion of training voice samples, the optimized language recognition model has higher recall and accuracy compared to the initial model. The optimized language recognition model can replace the initial language recognition model or the language recognition model optimized in the previous training round, effectively improving the accuracy of online user or user voice language tagging.

[0070] In one embodiment, the process provided by this solution of collecting voice data from target users whose language tags and social language tags are consistent to obtain training voice samples, and using the training voice samples to train the language recognition model, can be continuously and automatically carried out to achieve continuous automated iteration of the language recognition model and improve the accuracy of language recognition.

[0071] The above describes a process where a language recognition model is used to identify the language of the speech information to be recognized, yielding the language recognition result. Training speech samples for this model are obtained by collecting speech data from target users whose language identification labels and social language labels match. This allows for large-scale expansion of the training speech samples. Audio data from target users whose language identification and social language matches exhibit higher language confidence in multilingual scenarios, resulting in more reliable training speech samples, more diverse training content, and higher recognition accuracy for the trained language recognition model in complex linguistic environments. This effectively improves language recognition accuracy in complex multilingual scenarios. Furthermore, by filtering target users based on their language identification and social language labels, and ensuring consistency between their language identification and social language, the accuracy of the language recognition model in complex linguistic environments is further enhanced. Furthermore, it has achieved the construction of a low-cost, large-scale, and high-confidence business scenario language training dataset, which greatly improves the recognition capability of the language recognition model. At the same time, it can effectively realize the automatic labeling of user languages ​​on multilingual social platforms, providing a reliable guarantee for subsequent recommendation delivery, content review, content recognition and understanding and other scenarios.

[0072] Figure 5 This is a schematic diagram of the structure of a language recognition device provided in an embodiment of this application. (Reference) Figure 5 The language recognition device includes a voice acquisition module 51 and a language recognition module 52.

[0073] The speech acquisition module 51 is configured to acquire the speech information of the user to be identified; the language recognition module 52 is configured to input the speech information to be identified into the trained language recognition model, and perform language recognition on the speech information to be identified through the language recognition model to obtain the language recognition result corresponding to the user to be identified. The language recognition model is trained by pre-collected training speech samples, which are obtained by collecting speech data from target users whose recognition language labels and social language labels are consistent.

[0074] The above describes a method that uses a language recognition model to identify the language of the speech information to be identified, resulting in a language recognition result. The training speech samples used to train this model are obtained by collecting speech data from target users whose language tags and social language tags are consistent. This allows for large-scale data expansion of the training speech samples used to train the model. The audio data of target users whose language tags and social language tags are consistent have higher language confidence in multilingual scenarios, resulting in higher reliability of the training speech samples, more diverse training speech content, and higher recognition accuracy of the trained language recognition model in complex language environments. This effectively improves the accuracy of language recognition in complex multilingual scenarios.

[0075] In one possible embodiment, the language recognition device further includes a sample collection module configured to perform a training speech sample collection step. The sample collection module includes a language recognition unit, a social language unit, a user filtering unit, and the sample collection unit, wherein:

[0076] The language identification unit is configured to acquire initial speech samples from multiple sample users and determine the language identification label for each sample user based on the initial speech samples.

[0077] The social language unit is configured to acquire social relationship information of each sample user and determine the social language tag of each sample user based on the social relationship information.

[0078] The user filtering unit is configured to filter sample users based on the identified language tags and social language tags to obtain multiple target users;

[0079] The sample collection unit is configured to collect voice data from each target user to obtain training voice samples.

[0080] In one possible embodiment, when the user filtering unit filters sample users based on identified language tags and social language tags to obtain multiple target users, it is configured as follows:

[0081] Sample users whose identification language tags and social language tags match are selected as target users.

[0082] In one possible embodiment, the language identification unit is configured to determine the language identification label for each sample user based on the initial speech samples as follows:

[0083] Based on the initial speech samples, determine one or more combinations of device language labels, user language labels, and region language labels for each sample user. The user language label is obtained by performing language recognition on the initial speech samples based on the language recognition model to be trained.

[0084] The identification language label for each sample user is determined based on one or more combinations of device language labels, user language labels, and regional language labels.

[0085] In one possible embodiment, when determining the language identification label for each sample user based on one or more combinations of device language labels, user language labels, and region language labels, the language identification unit is configured as follows:

[0086] Based on the set tag type priority, the identification language tag for each sample user is determined from one or more combinations of device language tag, user language tag, and regional language tag.

[0087] In one possible embodiment, the social language unit is configured as follows when determining the social language tag of each sample user based on social relationship information:

[0088] Based on social relationship information, the social relationship user circle corresponding to each sample user is determined. The social relationship user circle records the associated users and associated languages ​​of the sample user.

[0089] Based on the number of users associated with different related languages ​​in the social relationship user circle, the social language tag of each sample user is determined.

[0090] In one possible embodiment, associated users in a social relationship user circle are determined based on one or more combinations of the sample users' audio and video call relationships, live interaction relationships, message chat relationships, and address book relationships.

[0091] It is worth noting that in the embodiments of the language recognition device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0092] This application also provides a language recognition device, which can integrate the language recognition apparatus provided in this application. Figure 6 This is a schematic diagram of the structure of a language recognition device provided in an embodiment of this application. (Reference) Figure 6 The language recognition device includes: an input device 63, an output device 64, a memory 62, and one or more processors 61; the memory 62 is used to store one or more programs; when one or more programs are executed by one or more processors 61, the one or more processors 61 implement the language recognition method provided in the above embodiments. The language recognition device, equipment, and computer provided above can be used to execute the language recognition method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0093] This application also provides a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the language recognition method provided in the above embodiments. Of course, the computer-executable instructions provided in this application are not limited to the language recognition method provided above, but can also perform related operations in the language recognition method provided in any embodiment of this application. The language recognition apparatus, device, and storage medium provided in the above embodiments can execute the language recognition method provided in any embodiment of this application. Technical details not described in detail in the above embodiments can be found in the language recognition method provided in any embodiment of this application.

[0094] Based on the above embodiments, this application also provides a computer program product. The technical solution of this application, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions to cause a computer device, mobile terminal, or processor therein to execute all or part of the steps of the language recognition method provided in the various embodiments of this application.

Claims

1. A language identification method characterized by, The method comprises: obtaining voice information to be recognized of a user to be recognized; inputting the voice information to be recognized into a trained language recognition model, performing language recognition on the voice information to be recognized through the language recognition model, and obtaining a language recognition result corresponding to the user to be recognized, wherein the language recognition model is obtained through training of pre-collected training voice samples, the training voice samples are obtained through voice data collection on a target user with consistent identification language label and social language label, the identification language label is a language label confirmed through language recognition on the target user, and the social language label is a language label determined based on social relations of the target user.

2. The language identification method of claim 1, wherein, The collection step of the training voice samples comprises: obtaining initial voice samples of a plurality of sample users, and determining identification language labels of the sample users based on the initial voice samples; obtaining social relation information of the sample users, and determining social language labels of the sample users based on the social relation information; screening the sample users based on the identification language labels and the social language labels, and obtaining a plurality of target users; collecting voice data of the target users to obtain training voice samples.

3. The language identification method of claim 2, wherein, The screening of the sample users based on the identification language labels and the social language labels to obtain a plurality of target users comprises: screening the sample users with consistent identification language labels and social language labels as target users.

4. The language identification method of claim 2, wherein, The determination of the identification language labels of the sample users based on the initial voice samples comprises: determining a combination of one or more of a device language label, a user language label and a region language label of the sample users based on the initial voice samples, wherein the user language label is obtained through language recognition on the initial voice samples by a language recognition model to be trained; determining the identification language labels of the sample users based on the combination of one or more of the device language label, the user language label and the region language label.

5. The language identification method of claim 4, wherein, The determination of the identification language labels of the sample users based on the combination of one or more of the device language label, the user language label and the region language label comprises: determining the identification language labels of the sample users from the combination of one or more of the device language label, the user language label and the region language label according to a set label type priority.

6. The language identification method of claim 2, wherein, The determination of the social language labels of the sample users based on the social relation information comprises: determining a social relation user circle corresponding to the sample users based on the social relation information, wherein the social relation user circle records associated users corresponding to the sample users and associated languages; determining the social language labels of the sample users based on the number of associated users of different associated languages in the social relation user circle.

7. The language identification method of claim 6, wherein, The associated users in the social relation user circle are determined based on a combination of one or more of audio / video call relations, live interaction relations, message chat relations and address book relations of the sample users.

8. A language identification apparatus characterized by comprising: The voice acquisition module and the language recognition module are included, wherein: The voice acquisition module is configured to acquire voice information to be recognized of a user to be recognized; The language recognition module is configured to input the voice information to be recognized into a trained language recognition model, recognize the language of the voice information to be recognized through the language recognition model, obtain a language recognition result corresponding to the user to be recognized, and train the language recognition model through pre-collected training voice samples, collect voice data of a target user whose recognition language label and social language label are consistent to obtain the training voice samples, the recognition language label is a language label confirmed by language recognition of the target user, and the social language label is a language label determined based on a social relationship of the target user.

9. A language identification device, characterized by Comprise: a memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the language recognition method of any one of claims 1-7.

10. A non-volatile storage medium for storing computer-executable instructions, characterized in that, The computer executable instructions, when executed by a computer processor, are used to perform the language recognition method of any one of claims 1-7.

11. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the language recognition method of any one of claims 1-7.

Images