A multi-jack voice control method and system based on natural language processing

Abstract

The embodiment of the present invention provides a multi-jack voice control method and system based on natural language processing, which belongs to the field of voice control technology. The method includes: obtaining relevant voice information of initial voice information under a first language type; performing voice clustering according to preset voice information and relevant voice information to obtain similar voice information; obtaining ideal voice information from a voice mapping table according to similar voice information; performing voice replacement according to the ideal voice information and relevant voice information to obtain target voice information; performing voice recognition on the target voice information to obtain initial text information; performing text correction on the initial text information to obtain target text information; performing keyword recognition on the target text information to obtain target keywords and target distribution information of target keywords; performing semantic classification on the target text information according to the target keywords and target distribution information to obtain target semantic types; controlling the multi-jack socket according to the target semantic types to obtain target control results.

Description

Technical Field

[0001] The present invention relates to the field of intelligent sockets and voice control technology, and in particular to a multi-socket voice control method and system based on natural language processing. Background Art

[0002] In the context of the continuous development of modern smart home devices, the functions of multi-socket sockets are becoming increasingly diversified, among which the voice control function brings great convenience to users. Through voice control, users can accurately realize the power-on and power-off operations of each socket when using a multi-socket socket. However, the prior art faces many challenges when performing on-off control of multi-socket sockets. Among them, the accuracy of voice recognition is particularly prominent. In real life, people in different regions have various accents and rich and diverse dialects. The differences in these accents and dialects will cause great trouble to voice control. When users use voices with accents or dialects to issue on-off commands to multi-socket sockets, it is often difficult for the voice recognition system to accurately parse these voice information, which greatly reduces the on-off control effect of the multi-socket socket, and cannot meet the user's needs for convenient and accurate control, which seriously affects the user's experience. Summary of the invention

[0003] The main purpose of the embodiments of the present invention is to provide a multi-jack voice control method and system based on natural language processing, aiming to solve the problem in the related art that when the user uses a voice with an accent or dialect to issue an on-off command to a multi-jack socket, the voice recognition effect is poor, and the on-off control effect of the multi-jack socket cannot meet the user's needs for convenient and precise control, thereby seriously affecting the user's experience.

[0004] In a first aspect, an embodiment of the present invention provides a multi-jack voice control method based on natural language processing, comprising:

[0005] Collecting initial voice information, and classifying the initial voice information by language type to obtain relevant voice information corresponding to the initial voice information under a first language type;

[0006] Determine preset voice information, and perform voice clustering based on the preset voice information and the related voice information to obtain similar voice information corresponding to the related voice information;

[0007] Obtaining ideal voice information corresponding to the relevant voice information in a second language type from a voice mapping table according to the similar voice information;

[0008] Performing voice replacement on the initial voice information according to the ideal voice information and the relevant voice information to obtain target voice information corresponding to the initial voice information;

[0009] Performing speech recognition on the target speech information according to a speech recognition model to obtain initial text information corresponding to the target speech information;

[0010] Performing text correction on the initial text information to obtain target text information corresponding to the target voice information;

[0011] Performing keyword recognition on the target text information to obtain target keywords and target distribution information corresponding to the target keywords;

[0012] Performing semantic classification on the target text information according to the target keywords and the target distribution information to obtain a target semantic type corresponding to the initial voice information;

[0013] The multi-jack socket is controlled according to the target semantic type to obtain a target control result corresponding to the initial voice information.

[0014] In a second aspect, an embodiment of the present invention provides a multi-jack voice control system based on natural language processing, comprising:

[0015] A speech analysis module, used for collecting initial speech information, and classifying the initial speech information into language types to obtain relevant speech information corresponding to the initial speech information in a first language type;

[0016] A voice clustering module, used to determine preset voice information, and perform voice clustering based on the preset voice information and the related voice information to obtain similar voice information corresponding to the related voice information;

[0017] A mapping processing module, used for obtaining, from a voice mapping table, the ideal voice information corresponding to the relevant voice information in a second language type according to the similar voice information;

[0018] A voice replacement module, used for performing voice replacement on the initial voice information according to the ideal voice information and the related voice information to obtain target voice information corresponding to the initial voice information;

[0019] A speech recognition module, used to perform speech recognition on the target speech information according to a speech recognition model to obtain initial text information corresponding to the target speech information;

[0020] A text correction module, used for performing text correction on the initial text information to obtain target text information corresponding to the target voice information;

[0021] A data identification module, used to perform keyword identification on the target text information to obtain target keywords and target distribution information corresponding to the target keywords;

[0022] A type classification module, used for semantically classifying the target text information according to the target keywords and the target distribution information to obtain a target semantic type corresponding to the initial voice information;

[0023] A result control module is used to control the multi-jack socket according to the target semantic type to obtain the corresponding target control result under the initial voice information.

[0024] The embodiment of the present invention provides a multi-jack voice control method based on natural language processing and a system thereof, the method comprising: by collecting initial voice information and performing language type classification, the language type to which the voice belongs can be accurately identified, thereby obtaining relevant voice information corresponding to the initial voice information under a first voice type, and then performing voice clustering according to preset voice information and relevant voice information, it is possible to find similar voice expressions in the voice to obtain similar voice information, and then according to the similar voice information, obtain ideal voice information corresponding to the relevant voice information under a second language type from a voice mapping table, thereby performing voice replacement on the initial voice information according to the ideal voice information and the relevant voice information to obtain target voice information corresponding to the initial voice information, thereby using a voice recognition model to recognize the target voice information to obtain initial text information The information is then used to correct the initial text information, and the errors that may occur during the speech recognition process can be corrected to obtain the target text information, thereby providing good support for the subsequent control of the multi-socket socket. The target keywords and corresponding target distribution information in the target text information are then obtained through keyword recognition, so that the target text information is semantically classified according to the target keywords and target distribution information, and the target semantic type of the initial speech information can be accurately judged, providing a basis for subsequent intelligent decision-making. Finally, the multi-socket socket is controlled according to the target semantic type, realizing the intelligent control of the intelligent device through voice commands. In this process, the user does not need to operate manually, but only needs to issue voice commands to conveniently control the socket. In addition, the user does not need to worry about the problem that his or her own language is not standard enough, thereby realizing convenient control of the multi-socket socket. It also solves the problem in the related art that when the user uses a voice with an accent or dialect to issue an on-off command to the multi-socket socket, the speech recognition effect is poor, thereby making the on-off control effect of the multi-socket socket unable to meet the user's needs for convenient and accurate control, thereby seriously affecting the user's experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A flowchart of a multi-jack voice control method based on natural language processing provided by an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of the module structure of a multi-jack voice control system based on natural language processing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0030] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0031] The embodiment of the present invention provides a multi-jack voice control method based on natural language processing and a system thereof. The multi-jack voice control method based on natural language processing can be applied to a terminal device, which can be an electronic device such as a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device. The terminal device can be a server or a server cluster.

[0032] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0033] Please refer to Figure 1 , Figure 1 A flowchart of a multi-jack voice control method based on natural language processing is provided in an embodiment of the present invention.

[0034] like Figure 1 As shown, the multi-jack voice control method based on natural language processing includes steps S101 to S109.

[0035] Step S101: collecting initial voice information, and classifying the initial voice information into language types to obtain relevant voice information corresponding to the initial voice information in a first language type.

[0036] Exemplarily, in order to reduce the difficulty of user parameter adjustment during the use of a multi-jack socket, the present application collects initial voice information corresponding to when the target user controls the multi-jack socket.

[0037] For example, when using a multi-jack socket in practice, since users come from different regions, each region has its own unique dialect and language habits, and different users also have their own unique language expressions, which leads to various language problems when outputting the initial voice information related to the operation of the multi-jack socket. Some users do not speak standard Mandarin, and their pronunciation may have a strong local accent; some users may not speak Mandarin at all and can only use local dialects to express; and some users may even have Mandarin and dialect cross-output during the communication process. These complex voice conditions undoubtedly bring huge challenges to voice recognition. If recognition is performed directly, it is easy to have recognition errors or inaccurate recognition problems, which in turn affects the accurate control of the multi-jack socket. Therefore, in order to effectively improve the accuracy of voice recognition, the present application firstly conducts an in-depth analysis of the characteristics, vocabulary, grammar, etc. of the collected initial voice information, and then classifies the language type to determine the specific language type to which each sub-voice information in the initial voice information belongs.

[0038] Exemplarily, after classification, the relevant voice information corresponding to the initial voice information under the first language type is obtained. The first language type refers to a non-mainstream language type. For example, it may be a dialect with distinct regional characteristics, such as Cantonese, Minnan dialect, Northeastern dialect, etc.; it may also be a language that belongs to the mainstream language type but has a distinct accent, such as Mandarin with Sichuan accent, Mandarin with Guangdong accent, etc. Therefore, the relevant voice information corresponding to the first language type is obtained from the initial voice information according to the specific language type.

[0039] Step S102: determine preset voice information, and perform voice clustering based on the preset voice information and the related voice information to obtain similar voice information corresponding to the related voice information.

[0040] Exemplarily, the preset voice information is voice information collected in advance when operating a multi-jack socket under the first language type, and then voice clustering is performed based on the preset voice information and related voice information according to a clustering algorithm, so that the remaining voice information of the cluster where the related voice information is located is determined as its corresponding similar voice information.

[0041] In some embodiments, the voice clustering according to the preset voice information and the related voice information to obtain similar voice information corresponding to the related voice information includes: performing discrete processing on the preset voice information to obtain first discrete data and performing discrete processing on the related voice information to obtain second discrete data; calculating the discrete distance between the first discrete data and the second discrete data according to the mutual correlation function; performing voice clustering on the preset voice information and the related voice information according to the discrete distance to obtain a target clustering result corresponding to the related voice information; obtaining an initial clustering cluster corresponding to the related voice information according to the target clustering result, and removing the related voice information from the initial clustering cluster to obtain a target clustering cluster; obtaining a target semantic type corresponding to each sub-voice information in the target clustering cluster under the first language type; performing data statistics on the target semantic type to obtain the maximum semantic type corresponding to the target clustering cluster; and obtaining the similar voice information corresponding to the related voice information according to the maximum semantic type and the first language type.

[0042] Exemplarily, the preset voice information is the voice information collected in advance when operating the multi-jack socket in the first language type, and then the preset voice information is segmented according to certain rules, such as dividing according to the phonemes, syllables or specific time windows of the voice, and the continuous voice signal is converted into a series of discrete data points, thereby obtaining the first discrete data. The same discretization rule is used to process the relevant voice information and convert it into the second discrete data. This allows the voice information to exist in a form that is easier to analyze and compare.

[0043] Exemplarily, a cross-correlation function is used to measure the similarity between the first discrete data and the second discrete data, thereby obtaining a discrete distance between them. The cross-correlation function can analyze the correlation between two discrete sequences at different positions, and the calculated discrete distance can reflect the degree of difference in features between the preset voice information and the related voice information. Thus, the preset voice information and the related voice information are clustered according to the calculated discrete distance. Voice information with a close distance will be classified into the same category, thereby obtaining a target clustering result corresponding to the related voice information. The clustering process can use common clustering algorithms, such as K-means clustering, etc., to determine the category to which the voice information belongs based on the discrete distance.

[0044] For example, the initial clustering cluster where the relevant speech information is located is found from the target clustering result, and then the relevant speech information is removed from this cluster to obtain the target clustering cluster. This step helps to remove the own information that may interfere with the subsequent analysis, so that the subsequent semantic analysis can focus more on other relevant speech.

[0045] Exemplarily, for each sub-speech information in the target cluster, the target semantic type corresponding to each sub-speech information under the first language type is identified by combining the first language type with natural language processing technology and a pre-trained semantic model, and then data statistics are performed on the target semantic types of all sub-speech information in the target cluster, and the frequency of occurrence of each semantic type is counted. By comparing the frequencies, the semantic type with the highest frequency, that is, the maximum semantic type, is found.

[0046] Exemplarily, when the preset voice information is collected in advance, in addition to knowing the language type corresponding to the preset voice information, the semantic type corresponding to each preset voice information is also known, and then based on the maximum semantic type and the first language type, the voice information matching the relevant voice information is filtered out from the preset voice information, and it is determined as the similar voice information corresponding to the relevant voice information.

[0047] Specifically, discrete processing converts continuous voice signals into discrete data, which makes it easier to extract the characteristic information of voice. By calculating discrete distances and cluster analysis, the similarities and differences between voices can be more accurately identified, which helps to tap the intrinsic characteristics of voice and provide a more reliable foundation for subsequent voice processing and recognition. In addition, by performing semantic analysis and data statistics on the target clustering clusters, the semantic type of voice information can be grasped more accurately. The determination of the maximum semantic type can help the system better understand the core meaning of the voice, avoid semantic misjudgments caused by interference from individual voice information, and improve the accuracy of semantic analysis, thereby providing good support for the subsequent more accurate recognition of user voice commands in the voice control scenario of multi-jack sockets.

[0048] In some embodiments, the method of performing voice clustering on the preset voice information and the related voice information according to the discrete distance to obtain a target clustering result corresponding to the related voice information includes: setting a distance parameter, and determining a first density corresponding to the first discrete data and a second density corresponding to the second discrete data according to the discrete distance and the distance parameter; determining a first offset distance corresponding to the first discrete data and a second offset distance corresponding to the second discrete data according to the first density and the second density; determining a first probability corresponding to the first discrete data as a cluster center according to the first density and the first offset distance; determining a second probability corresponding to the second discrete data as a cluster center according to the second density and the second offset distance; arranging data according to the first probability and the second probability to obtain a probability sequence, and determining the number of clusters corresponding to voice clustering of the preset voice information and the related voice information according to the probability sequence; and performing voice clustering according to the number of clusters combined with the discrete distance to obtain the target clustering result.

[0049] Exemplarily, the distance parameter is a pre-set threshold value, which is used to measure the distance range within which discrete data points can be considered to be related to each other. The setting of this parameter can be adjusted in combination with the business scenario and data characteristics of the multi-socket socket. For the first discrete data and the second discrete data, the number of data points in the neighborhood with each data point as the center and the distance parameter as the radius is calculated respectively, and this number is used to represent the density of the data point. Specifically, if there are more other data points around a data point within the distance parameter range, then its density is high; otherwise, the density is low.

[0050] For example, for each data point in the first discrete data, find the point closest to it among the data points with a higher density than it, calculate the distance between the two points, and this distance is the first offset distance of the data point. Similarly, for each data point in the second discrete data, find the point closest to it among the data points with a higher density than it, calculate the distance between the two points, and obtain the second offset distance of the data point.

[0051] Exemplarily, the first probability of the first discrete data point being the cluster center is determined by combining the first density and the first offset distance. Generally speaking, a data point with a high density and a large offset distance is more likely to be a cluster center, and the first probability of the data point being the cluster center is obtained by a specific formula (such as the product of the two or a weighted calculation). Using the same method, the second probability of the second discrete data point being the cluster center is calculated based on the second density and the second offset distance.

[0052] Exemplarily, the first probability and the second probability are combined and arranged from large to small according to the probability value to form a probability sequence. By observing the probability sequence, the number of clusters can usually be determined according to the changing trend of the probability value. For example, when there is an obvious "fault" in the probability value, that is, a certain probability value has a significant decrease compared with the previous probability value, the number of high-probability data points before this "fault" can be used as the number of clusters.

[0053] Exemplarily, according to the determined number of clusters, combined with the previously calculated discrete distance, a clustering algorithm such as a density-based clustering algorithm is used to cluster the preset voice information and the related voice information, and finally a target clustering result is obtained.

[0054] Specifically, traditional clustering methods often require people to pre-specify the number of clusters, which may lead to poor clustering results in practical applications due to insufficient understanding of the data. By calculating the probability of a data point being a cluster center and determining the number of clusters, the clustering process can adapt to the distribution characteristics of the data, thereby more accurately dividing different clusters and avoiding the problem of too many or too few clusters. In addition, when determining the cluster center, the density and offset distance of the data point are taken into account at the same time, which can more comprehensively reflect the position and distribution of the data point in the data space, so that the true cluster center can be more accurately identified, improving the accuracy of clustering, and then in the voice control scenario of multi-jack sockets, it provides good support for the subsequent more accurate identification of different types of user voice commands.

[0055] In some embodiments, the method of arranging data according to the first probability and the second probability to obtain a probability sequence, and determining the number of clusters corresponding to voice clustering of the preset voice information and the related voice information according to the probability sequence, includes: arranging the first probability and the second probability in descending order to obtain the probability sequence; obtaining any initial data in the probability sequence, and obtaining left probability data corresponding to the left side of the initial data and right probability data corresponding to the right side of the initial data from the probability sequence; performing slope calculation according to the initial data, the left probability data, and the right probability data to obtain a target value corresponding to the initial data; obtaining the maximum value corresponding to the target value, and determining the position information corresponding to the maximum value according to the initial data, and determining the number of clusters corresponding to voice clustering of the preset voice information and the related voice information according to the position information; wherein the target value is obtained according to the following formula:

[0056]

[0057] in, represents the target value corresponding to the i-th initial data, represents the i-th initial data, represents the left probability data corresponding to the i-th initial data, Represents the right probability data corresponding to the i-th initial data.

[0058] Exemplarily, the first probability and the second probability calculated previously are combined and arranged in descending order from large to small to obtain an ordered probability sequence. Each element in this sequence represents the probability of the corresponding discrete data point as the cluster center. The larger the probability value, the more likely the data point is to become the cluster center. A data is randomly selected from the ordered probability sequence as the initial data. Determine the position of the initial data in the probability sequence, and then find the data immediately to its left, that is, the left probability data; at the same time, find the data immediately to its right, that is, the right probability data.

[0059] Exemplarily, the slope is calculated based on the initial data, the left probability data, and the right probability data to obtain the target value corresponding to the initial data, and the target value is used to measure the rate of change of the probability value near the point. This target value reflects the degree of change of the probability sequence at the initial data point, wherein the target value can be obtained according to the following formula:

[0060]

[0061] in, represents the target value corresponding to the i-th initial data, represents the i-th initial data, Represents the left probability data corresponding to the i-th initial data, Represents the right probability data corresponding to the i-th initial data.

[0062] Exemplarily, after calculating the target value corresponding to each initial data in the probability sequence, find the maximum value among all the target values. Determine the position information of the initial data corresponding to this maximum value in the probability sequence, so as to determine the number of clusters corresponding to the preset voice information and the related voice information when performing voice clustering based on the position information. Generally speaking, the probability value before this position is relatively high and the change is relatively slow, while the probability value after this position shows an obvious downward trend, so this position can be used as a key node for dividing different clusters, and the serial number corresponding to this position can be used as the number of clusters.

[0063] Specifically, by calculating the target value, we can keenly capture the changing trend of the probability value in the probability sequence. The location where the probability value changes dramatically usually means that there are obvious differences in the data distribution, corresponding to different clustering boundaries. Therefore, determining the number of clusters based on the maximum value of the target value can more accurately divide the data into different clusters, improve the quality of clustering, and make the subsequent clustering results more reliable and repeatable.

[0064] Step S103: Obtain ideal voice information corresponding to the relevant voice information in a second language type from a voice mapping table according to the similar voice information.

[0065] For example, the second language type refers to a mainstream language type that is widely used and has a standard and normative form. It is a language form commonly used in many scenarios such as formal communication, education, and media communication. For example, in the scope of Chinese, Mandarin is a typical second language type, and in the context of English, the second language type is standard British English or standard American English.

[0066] Exemplarily, in order to achieve a specific speech processing goal, a speech mapping table is constructed in advance. This speech mapping table is established after a large amount of data collection, sorting and analysis, and it records the corresponding relationship between the speech information corresponding to the operation of various multi-jack sockets. In this mapping table, the association between similar speech information in different language types is stored. When it is necessary to process related speech information, the speech mapping table can be used to find the speech information corresponding to the similar speech information in the second language type, and then the speech information is determined as the ideal speech information corresponding to the related speech information in the second language type.

[0067] Step S104: performing voice replacement on the initial voice information according to the ideal voice information and the related voice information to obtain target voice information corresponding to the initial voice information.

[0068] Exemplarily, the relevant voice information in the initial voice information is replaced with the ideal voice information, thereby obtaining the target voice information corresponding to the initial voice information in the second language type.

[0069] Step S105: performing speech recognition on the target speech information according to a speech recognition model to obtain initial text information corresponding to the target speech information.

[0070] Exemplarily, the speech recognition model can be optimized for voice commands that may appear in multi-socket scenarios, such as end-to-end speech recognition models and machine learning models obtained through voice training for related commands such as "open socket 1", "close socket 3", and "close socket 2 at a certain time". The pre-trained speech recognition model is then deployed to a terminal device that is connected to the multi-socket communication network, and after obtaining the target voice information, the speech recognition model is used to perform speech recognition on the target voice information to obtain the initial text information corresponding to the target voice information. For example, in a multi-socket scenario, if the customer says "open the first socket" in the target voice information, after model recognition and decoding, the output initial text information may be "open the first socket".

[0071] In some embodiments, obtaining the speech recognition model includes: obtaining the complete speech information corresponding to the multi-jack socket in the second language type and the relevant text information corresponding to the complete speech information; performing word splitting on the relevant text information to obtain a word sequence, and counting the frequency information corresponding to each sub-word in the word sequence and the word length corresponding to each sub-word; merging the sub-words according to the word length and the frequency information to obtain a target coding table corresponding to the relevant text information; using the data sampling layer of the speech recognition model to downsample the complete speech information to obtain a first processing result corresponding to the complete speech information; using the linear processing layer of the speech recognition model to linearly process the first processing result to obtain a second processing result; using the data coding layer of the speech recognition model to perform coding on the second processing result. The target coding table is combined to perform data encoding to obtain a target coding result; the first decoding layer of the speech recognition model is used to decode the target coding result to obtain a first decoding result; the second decoding layer of the speech recognition model is used to decode the target coding result to obtain a second decoding result; the loss calculation layer of the speech recognition model is used to obtain a first loss result according to the first decoding result and a second loss result according to the second decoding result to perform loss calculation to obtain a second loss result; the loss fusion layer of the speech recognition model is used to perform loss fusion according to the first loss result and the second loss result to obtain a target loss result; the speech recognition model is adjusted according to the target loss result to obtain the model parameters corresponding to the target loss result when the preset conditions are met, and the speech recognition model is obtained according to the model parameters.

[0072] For example, in the use scenario of a multi-socket, a large amount of voice information related to the operation of the multi-socket is collected. The voice information is expressed in a second language type, such as "Turn on the seconds socket" expressed in standard English. At the same time, the existing voice recognition is used to preliminarily convert these voices into corresponding text information, and then the relevant text information obtained is split into words to form a word sequence. For example, "Turn on the second socket" is split into "Turn", "on", "the", "second", and "socket". Then the frequency information of each sub-word is counted, that is, the number of times the sub-word appears in all text information, and the length of each sub-word is recorded, for example, the length of "Turn" is 4. The merging operation is performed according to the length and frequency information of the sub-word. For those sub-words with high frequency of occurrence and short length, merging can be given priority to reduce the complexity of encoding. For example, "on the" often appear together and can be merged into one encoding unit. Finally, a target encoding table corresponding to the relevant text information is formed, which records the corresponding relationship between the merged word unit and the encoding.

[0073] Exemplarily, the complete voice information is downsampled using the data sampling layer of the speech recognition model. Downsampling is to reduce the number of sampling points of the voice signal and reduce the dimension of the data. For example, originally 16,000 points were sampled per second, and after downsampling, 8,000 points may be sampled per second. This can reduce the amount of data for subsequent processing and improve processing efficiency. After downsampling, the first processing result corresponding to the complete voice information is obtained. The first processing result is then input into the linear processing layer of the speech recognition model. The linear processing layer performs a linear transformation on the input data, and adjusts the feature representation of the data through operations such as matrix multiplication to make it more conducive to subsequent encoding processing. After linear processing, the second processing result is obtained.

[0074] Exemplarily, the data encoding layer of the speech recognition model is used to combine the second processing result with the previously generated target encoding table for data encoding. The encoding process is to convert the speech features into corresponding encoding sequences according to the target encoding table for subsequent decoding and recognition. Finally, the target encoding result is obtained.

[0075] Exemplarily, the target encoding result is decoded by using the first decoding layer and the second decoding layer of the speech recognition model. The two decoding layers may adopt different decoding strategies or algorithms to obtain different decoding results, namely, the first decoding result and the second decoding result. For example, the first decoding layer may adopt a greedy decoding strategy, and the second decoding layer may adopt a beam search decoding strategy.

[0076] Exemplarily, the loss calculation layer of the speech recognition model calculates the loss according to the first decoding result and the second decoding result respectively. Loss is an indicator to measure the difference between the decoding result and the true label. Commonly used loss functions include cross entropy loss, etc. The first loss result and the second loss result are obtained by calculation, and then the two loss results are comprehensively calculated to obtain the final second loss result.

[0077] For example, the loss fusion layer fuses the first loss result and the second loss result to obtain the target loss result. Loss fusion can use methods such as weighted averaging to assign different weights according to the reliability of different decoding results to more accurately reflect the performance of the model.

[0078] Exemplarily, the parameters of the speech recognition model are adjusted according to the target loss result. Using optimization algorithms, such as stochastic gradient descent, adaptive moment estimation, etc., the parameters of the model are continuously updated so that the target loss result is gradually reduced. When the target loss result meets the preset conditions, such as the loss value is less than a certain threshold or the loss no longer decreases significantly within a certain number of iterations, the parameter adjustment is stopped. The model parameters obtained at this time are the final model parameters, and the optimized speech recognition model can be determined based on these parameters.

[0079] Specifically, the target coding table is used for data encoding, and the commonly used word combinations are combined for encoding, which reduces the complexity of encoding and further improves the efficiency of encoding and decoding. In addition, two different decoding layers are used for decoding, and the decoding results are calculated and fused for loss, so that the model can comprehensively consider the advantages of different decoding strategies, improve the recognition accuracy and robustness of the model in different voice scenarios, and then the optimized speech recognition model can more accurately recognize the voice commands related to the multi-jack socket, reduce recognition errors, and provide good support for the subsequent user to quickly control the multi-jack socket.

[0080] Step S106: Perform text correction on the initial text information to obtain target text information corresponding to the target voice information.

[0081] Exemplarily, a large amount of text data in the field of multi-jack socket control related to the target voice information is collected from multiple channels, and then common text error types such as spelling errors, grammatical errors, misuse of homophones, etc. are analyzed, and then these error instances are collected to form an error pattern library, so as to quickly identify and handle similar errors in the correction process. Thus, the initial text information is scanned using a spelling checker, and then each word in the text is compared with the correct word in the dictionary to handle spelling errors. In addition, the grammar rule engine can also be used to analyze the sentence structure of the initial text. The grammar rule engine contains a large number of grammar rules, so as to identify the sentence parts that do not conform to the grammar rules and give corresponding correction suggestions, so as to perform text correction on the initial text information to obtain the target text information corresponding to the target voice information.

[0082] Step S107: Perform keyword recognition on the target text information to obtain target keywords and target distribution information corresponding to the target keywords.

[0083] Exemplarily, a semantic analysis model related to multi-outlet socket control is trained, and then semantic analysis is performed on the target text information according to the semantic analysis model so as to use keywords in the target text information that are highly relevant to multi-outlet socket control as target keywords.

[0084] Exemplarily, the position information of the target keyword in the target text information is obtained, and then the target distribution information corresponding to the target keyword in the target text information is determined according to the position information.

[0085] Step S108: semantically classify the target text information according to the target keywords and the target distribution information to obtain a target semantic type corresponding to the initial voice information.

[0086] For example, in the field of multi-jack sockets, clear definitions and boundaries are formulated for each semantic category. The semantic types included in the field of multi-jack sockets may be using double and triple sockets at the same time, using three double sockets at the same time, using only triple sockets, etc., and then sorting out the typical keyword set under each semantic category, as well as the position distribution information of each keyword under the semantic category, so as to use the semantic similarity algorithm to compare the target text information with the standard text under each semantic category using the target keyword and the target distribution information. If the similarity exceeds a certain threshold, the target text is classified into the category, and then the target semantic type corresponding to the initial voice information is obtained.

[0087] In some embodiments, the semantic classification of the target text information according to the target keyword and the target distribution information to obtain the target semantic type corresponding to the initial voice information includes: obtaining the associated semantic type corresponding to the multi-jack socket, the initial keyword corresponding to the associated semantic type, and the initial distribution information corresponding to the initial keyword; performing similarity calculation according to the target keyword and the initial keyword to obtain a text similarity value and performing distribution similarity calculation according to the initial distribution information and the target distribution information to obtain a distribution similarity value; determining the target correlation between the initial voice information and the associated semantic type according to the text similarity value and the distribution similarity value; and obtaining the target semantic type corresponding to the initial voice information from the associated semantic type according to the target correlation.

[0088] Exemplarily, clear definitions and boundaries are established for each associated semantic category in the field of multi-socket sockets. The semantic types included in the field of multi-socket sockets may be using double sockets and triple sockets at the same time, using three double sockets at the same time, using only triple sockets, etc., thereby obtaining the associated semantic types corresponding to the multi-socket sockets, the initial keywords corresponding to the associated semantic types, and the initial distribution information corresponding to the initial keywords.

[0089] For example, the text under each associated semantic type is processed to remove noise information such as stop words and punctuation marks, and then the initial keywords that can represent the semantic type are extracted through statistical methods or rule-based methods, and the position of each initial keyword is recorded for the extracted initial keywords. Thus, the initial distribution information corresponding to the initial keywords is obtained by analyzing the position distribution of the keywords.

[0090] Exemplarily, after the target keyword and the initial keyword are converted into vectors using a word embedding model such as Word2Vec, the text similarity value between the target keyword and the initial keyword is calculated using a similarity measurement method such as edit distance, cosine similarity, etc. KL divergence is used to measure the similarity between distribution feature vectors using target distribution information and initial distribution information to obtain a distribution similarity value.

[0091] Exemplarily, weights are set for the text similarity value and the distribution similarity value. For example, if the semantic matching degree of keywords is more important, a higher weight can be assigned to the text similarity value; if the distribution of keywords is considered to be more important for the judgment of semantic association, the weight of the distribution similarity value can be increased, so that the text similarity value and the distribution similarity value are weighted and summed according to the set weights to obtain the target association between the initial voice information and the associated semantic type.

[0092] Exemplarily, the target relevance between the initial speech information and all associated semantic types is sorted and arranged in descending order, and then the associated semantic type with the highest target relevance is selected as the target semantic type corresponding to the initial speech information. If the highest relevance is lower than a preset threshold, further analysis or manual intervention may be required to determine the appropriate semantic type.

[0093] In some embodiments, the determining the target correlation between the initial voice information and the associated semantic type according to the text similarity value and the distribution similarity value includes: determining a fusion parameter according to the text similarity value and the distribution similarity value, and performing data fusion on the text similarity value and the distribution similarity value according to the fusion parameter to obtain a target similarity value; obtaining a maximum similarity value corresponding to the target keyword from the target similarity value, and obtaining a word frequency weight corresponding to the target keyword from the initial distribution information; determining the target correlation between the initial voice information and the associated semantic type according to the target similarity value and the word frequency weight; wherein the target correlation is obtained according to the following formula:

[0094]

[0095]

[0096] in, represents the target relevance between the initial voice information and the i-th associated semantic type; all represents the target number corresponding to the target keyword, represents the word frequency weight corresponding to the jth target keyword, num represents the target number corresponding to the initial keyword, represents the fusion parameter, represents the text similarity value between the j-th target keyword and the x-th initial keyword, represents the distribution similarity value between the x-th initial distribution information and the j-th target distribution information; exp represents an exponential function, and max represents obtaining the maximum value.

[0097] Exemplarily, the fusion parameter is obtained using the following formula according to the text similarity value and the distribution similarity value:

[0098]

[0099] in, represents the fusion parameter, represents the text similarity value between the j-th target keyword and the x-th initial keyword, represents the distribution similarity value between the x-th initial distribution information and the j-th target distribution information; exp represents an exponential function.

[0100] Exemplarily, the text similarity value and the distribution similarity value are weighted and summed according to the fusion parameter to obtain the target similarity value, and then the target similarity values ​​corresponding to all target keywords are arranged from large to small. After the sorting is completed, the value ranked first is selected as the maximum similarity value corresponding to the target keyword.

[0101] Exemplarily, the frequency of occurrence of the target keyword is counted in the initial distribution information. The word frequency can be calculated by the ratio of the number of times the target keyword appears in the relevant text to the total number of words in the text. Thus, the word frequency weight is determined for the target keyword according to the size of the word frequency. Thus, the target relevance is obtained according to the following formula:

[0102]

[0103] in, represents the target relevance between the initial voice information and the i-th associated semantic type; all represents the target number corresponding to the target keyword, represents the word frequency weight corresponding to the jth target keyword, num represents the target number corresponding to the initial keyword, represents the fusion parameter, represents the text similarity value between the j-th target keyword and the x-th initial keyword, represents the distribution similarity value between the x-th initial distribution information and the j-th target distribution information; exp represents an exponential function, and max represents obtaining the maximum value.

[0104] For example, the target relevance comprehensively considers the similarity of keywords and their importance in the text, and can more accurately reflect the degree of relevance between the initial voice information and the associated semantic type. Then, according to the calculated target relevance, the relevance between the initial voice information and the associated semantic type can be evaluated. The higher the relevance, the stronger the correlation between the two.

[0105] Step S109: Control the multi-jack socket according to the target semantic type to obtain a target control result corresponding to the initial voice information.

[0106] Exemplarily, the corresponding control instructions are obtained from the database according to the target semantic type. For example, the target semantic type "turn on" corresponds to the control instruction "send a power on signal to the socket"; "turn off" corresponds to "send a power off signal to the socket"; "timed turn on" corresponds to "set the socket to turn on the power at a specified time"; "adjust power" corresponds to "send a power adjustment signal to the socket", so that these correspondences are stored in the database, which facilitates the subsequent search for the corresponding control instructions according to the target semantic type.

[0107] Exemplarily, after obtaining the control instruction corresponding to the target semantic type, ensure that the multi-socket socket is connected to the home network via Wi-Fi, and the smart device (such as a mobile phone, smart speaker) is also connected to the same network, so as to ensure that the two can interact with each other, and then send the found control instruction to the multi-socket socket through the communication interface. For example, the smart speaker sends the "turn on" instruction to the socket in a specific protocol format (such as JSON format) through the Wi-Fi network. After receiving the control instruction, the multi-socket socket parses the instruction content and performs the corresponding operation. In the example of the "turn on" instruction, the control circuit inside the socket will be closed to power the electrical equipment connected to the socket, and then the corresponding operation result under the control quality will be determined as the target control result corresponding to the initial voice information.

[0108] For example, if the initial voice message is "turn off the multi-socket in the bedroom at 10 pm", the target semantic type is "timed shutdown", and then the relevant time is found from the target text message, and then the instruction corresponding to "timed shutdown" is found from the database, "set the socket to turn off the power at the specified time", so that the smart device sends the shutdown instruction containing the time information of 10 pm to the multi-socket in the bedroom. After receiving the instruction, the socket sets the internal timer to automatically cut off the power at 10 pm.

[0109] In some embodiments, before collecting the initial voice information, the method further includes: collecting wake-up voice information, and performing voice classification on the wake-up voice information to obtain a voice classification result corresponding to the wake-up voice information; when the voice classification result is a preset type, opening a voice collection module corresponding to the multi-jack socket so that the initial voice information is collected according to the voice collection module.

[0110] Exemplarily, a speech classification model such as a support vector machine (SVM) or a neural network (such as a convolutional neural network (CNN) or a recurrent neural network (RNN)) is determined, and a large amount of labeled speech data is used to train the model so that the model can learn the characteristic patterns of different types of speech. For example, a data set containing wake-up word speech and non-wake-up word speech is prepared, and which ones are preset types of wake-up speech and which ones are not are marked, and then the classification model is trained with these data to obtain a speech classification model.

[0111] Exemplarily, the voice collection device is installed in a suitable position to ensure that the surrounding voices can be clearly collected, and then the feature extraction operation is performed on the collected wake-up voice information. The extracted wake-up voice information features are input into the trained voice classification model. The model performs classification prediction based on the learned feature pattern, outputs the voice classification result corresponding to the wake-up voice information, and determines whether the voice belongs to the preset type.

[0112] Exemplarily, after obtaining the voice classification result, it is judged. If the result is displayed as a preset type, it means that the user has issued a valid wake-up command. For example, the preset type is a voice containing a specific wake-up word "socket assistant". When the classification result shows that the wake-up voice contains "socket assistant", the condition is met, and a start signal is sent to the voice collection module corresponding to the multi-socket socket through a preset communication method (such as Wi-Fi, Bluetooth, ZigBee, etc.). The multi-socket socket needs to establish a stable communication connection with the control end (such as a smart speaker, a mobile phone, etc.), and the control end sends the start instruction to the voice collection module in a specific protocol format. Thus, after the voice collection module receives the start signal, it starts working. It starts to continuously monitor the surrounding voice signals as before to collect wake-up voice information. Once sound input is detected, audio data is collected and converted into digital signals as initial voice information for subsequent processing.

[0113] In some embodiments, after obtaining the target control result, the method further includes: obtaining a target jack state corresponding to the multi-jack socket according to the target control result; sending the target jack state to a user terminal communicatively connected to the multi-jack socket according to a communication module corresponding to the multi-jack socket, so that the target user obtains the target jack state corresponding to the multi-jack socket according to the user terminal.

[0114] Exemplarily, the target control result is the result of a previous control operation on a multi-socket socket, such as turning on a certain socket, turning off a certain socket, adjusting the power of a certain socket, etc. For example, the target control result may be "turning on socket 3" or "adjusting the power of socket 2 to 500 watts". Thus, a mapping relationship between the target control result and the socket state is established. For a multi-socket socket, each socket usually has several basic states, such as turned on, turned off, different power levels, etc. According to the target control result, the corresponding socket state is determined.

[0115] For example, if the target control result is to turn a socket on or off, the state of the socket will be changed to on or off accordingly. For example, if the control result is "turn off socket 1", the state of socket 1 is off. For the control result of adjusting power, in addition to the on state, the specific power value of the socket must be recorded. For example, if the control result is "adjust the power of socket 4 to 800 watts", the state of socket 4 is on and the power is 800 watts.

[0116] Exemplarily, a multi-socket socket is equipped with a communication module, and common ones include wifi, Bluetooth, ZigBee, etc. The communication module is selected according to the actual application scenario and requirements. For example, in a home environment, a wifi communication module is more commonly used because the home usually has mature wifi network coverage, and then the selected communication module is configured so that it can establish a stable communication connection with the user terminal, so as to encapsulate the target socket state for accurate transmission during the communication process. According to the requirements of the communication protocol, the status information of each socket (such as the on / off state, power value, etc.) is organized into a data packet in a certain format, so that the encapsulated data packet is sent to the user terminal through the configured communication module.

[0117] Exemplarily, a user terminal (such as a mobile phone, tablet computer, etc.) receives a data packet from a multi-socket socket through a corresponding application or client software. The communication interface of the user terminal receives the wireless signal and converts it into digital data. Then, the data packet is parsed according to the same communication protocol and data format to extract the status information of each socket. The user terminal displays the parsed socket status information to the target user in an intuitive manner. For example, in a mobile phone application, the layout of the multi-socket socket can be displayed in a graphical interface, each socket is represented by an icon, and its on / off status and power value are described by color or text. The user can understand the real-time status of each socket at a glance, and further control or manage the socket based on this information.

[0118] See also Figure 2 , Figure 2A multi-jack voice control system 200 based on natural language processing is provided in an embodiment of the present application. The multi-jack voice control system 200 based on natural language processing includes a voice analysis module 201, a voice clustering module 202, a mapping processing module 203, a voice replacement module 204, a voice recognition module 205, a text correction module 206, a data recognition module 207, a type classification module 208, and a result control module 209, wherein the voice analysis module 201 is used to collect initial voice information, and classify the initial voice information into language types to obtain relevant voice information corresponding to the initial voice information under a first language type; the voice clustering module 202 is used to determine preset voice information, and perform voice clustering according to the preset voice information and the relevant voice information to obtain similar voice information corresponding to the relevant voice information; the mapping processing module 203 is used to obtain, from a voice mapping table, an ideal voice information corresponding to the relevant voice information under a second language type according to the similar voice information information; a voice replacement module 204, used to perform voice replacement on the initial voice information according to the ideal voice information and the related voice information to obtain the target voice information corresponding to the initial voice information; a voice recognition module 205, used to perform voice recognition on the target voice information according to the voice recognition model to obtain the initial text information corresponding to the target voice information; a text correction module 206, used to perform text correction on the initial text information to obtain the target text information corresponding to the target voice information; a data recognition module 207, used to perform keyword recognition on the target text information to obtain the target keyword and the target distribution information corresponding to the target keyword; a type classification module 208, used to perform semantic classification on the target text information according to the target keyword and the target distribution information to obtain the target semantic type corresponding to the initial voice information; a result control module 209, used to control the multi-jack socket according to the target semantic type to obtain the corresponding target control result under the initial voice information.

[0119] In some implementations, the multi-jack voice control system 200 based on natural language processing can be applied to a terminal device.

[0120] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the multi-jack voice control system 200 based on natural language processing described above can refer to the corresponding process in the aforementioned multi-jack voice control method embodiment based on natural language processing, and will not be repeated here.

[0121] An embodiment of the present invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any step of the multi-jack voice control method based on natural language processing provided in the description of the embodiment of the present invention.

[0122] The storage medium may be an internal storage unit of the terminal device described in the foregoing embodiment, such as a hard disk or memory of the terminal device. The storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc., equipped on the terminal device.

[0123] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0124] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0125] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multi-socket voice control method based on natural language processing, characterized in that: The method comprises: Collecting initial voice information, and classifying the initial voice information by language type to obtain relevant voice information corresponding to the initial voice information under a first language type; Determine preset voice information, and perform voice clustering based on the preset voice information and the related voice information to obtain similar voice information corresponding to the related voice information; Obtaining ideal voice information corresponding to the relevant voice information in a second language type from a voice mapping table according to the similar voice information; Performing voice replacement on the initial voice information according to the ideal voice information and the relevant voice information to obtain target voice information corresponding to the initial voice information; Performing speech recognition on the target speech information according to a speech recognition model to obtain initial text information corresponding to the target speech information; Performing text correction on the initial text information to obtain target text information corresponding to the target voice information; Performing keyword recognition on the target text information to obtain target keywords and target distribution information corresponding to the target keywords; Performing semantic classification on the target text information according to the target keywords and the target distribution information to obtain a target semantic type corresponding to the initial voice information; Controlling the multi-jack socket according to the target semantic type to obtain a target control result corresponding to the initial voice information; The step of performing voice clustering according to the preset voice information and the related voice information to obtain similar voice information corresponding to the related voice information includes: Performing discrete processing on the preset voice information to obtain first discrete data and performing discrete processing on the related voice information to obtain second discrete data; Calculate the discrete distance between the first discrete data and the second discrete data according to the cross-correlation function; Performing voice clustering on the preset voice information and the related voice information according to the discrete distance to obtain a target clustering result corresponding to the related voice information; Obtaining an initial clustering cluster corresponding to the relevant speech information according to the target clustering result, and removing the relevant speech information from the initial clustering cluster to obtain a target clustering cluster; Obtaining a target semantic type corresponding to each sub-speech information in the target cluster under the first language type; Performing data statistics on the target semantic type to obtain the corresponding maximum semantic type in the target cluster; The similar voice information corresponding to the related voice information is obtained according to the maximum semantic type and the first language type.

2. The method according to claim 1, characterized in that The performing voice clustering on the preset voice information and the related voice information according to the discrete distance to obtain a target clustering result corresponding to the related voice information includes: Setting a distance parameter, and determining a first density corresponding to the first discrete data and a second density corresponding to the second discrete data according to the discrete distance and the distance parameter; Determine a first offset distance corresponding to the first discrete data and a second offset distance corresponding to the second discrete data according to the first density and the second density; Determine a first probability corresponding to the first discrete data being a cluster center according to the first density and the first offset distance; Determine a second probability corresponding to the second discrete data being a cluster center according to the second density and the second offset distance; Arranging data according to the first probability and the second probability to obtain a probability sequence, and determining the number of clusters corresponding to voice clustering of the preset voice information and the related voice information according to the probability sequence; Speech clustering is performed according to the cluster quantity in combination with the discrete distance to obtain the target clustering result.

3. The method according to claim 2, characterized in that The step of arranging data according to the first probability and the second probability to obtain a probability sequence, and determining the number of clusters corresponding to voice clustering of the preset voice information and the related voice information according to the probability sequence, comprises: Arrange the first probability and the second probability in descending order to obtain the probability sequence; Obtain any initial data in the probability sequence, and obtain left probability data corresponding to the left side of the initial data and right probability data corresponding to the right side of the initial data from the probability sequence; Perform slope calculation according to the initial data, the left probability data and the right probability data to obtain a target value corresponding to the initial data; Obtaining a maximum value corresponding to the target value, determining position information corresponding to the maximum value according to the initial data, and determining the number of clusters corresponding to voice clustering of the preset voice information and the related voice information according to the position information; The target value is obtained according to the following formula: in, represents the target value corresponding to the i-th initial data, represents the i-th initial data, represents the left probability data corresponding to the i-th initial data, Represents the right probability data corresponding to the i-th initial data.

4. The method according to claim 1, characterized in that: Obtaining the speech recognition model includes: Obtaining complete voice information corresponding to the multi-jack socket in the second language type and related text information corresponding to the complete voice information; Perform word splitting on the relevant text information to obtain a word sequence, and count the frequency information corresponding to each sub-word in the word sequence and the word length corresponding to each sub-word; Merging the sub-words according to the word length and the frequency information to obtain a target coding table corresponding to the relevant text information; Downsampling the complete speech information using a data sampling layer of a speech recognition model to obtain a first processing result corresponding to the complete speech information; Using the linear processing layer of the speech recognition model to perform linear processing on the first processing result to obtain a second processing result; Using the data encoding layer of the speech recognition model, the second processing result is combined with the target encoding table to perform data encoding to obtain a target encoding result; Using the first decoding layer of the speech recognition model to decode the target encoding result to obtain a first decoding result; Using the second decoding layer of the speech recognition model to decode the target encoding result to obtain a second decoding result; Using the loss calculation layer of the speech recognition model to obtain a first loss result according to the first decoding result and a second loss result according to the second decoding result, a loss calculation is performed to obtain a second loss result; Using the loss fusion layer of the speech recognition model to perform loss fusion according to the first loss result and the second loss result to obtain a target loss result; According to the target loss result, the parameters of the speech recognition model are adjusted to obtain the model parameters corresponding to when the target loss result meets the preset conditions, and the speech recognition model is obtained according to the model parameters.

5. The method according to claim 1, characterized in that The performing semantic classification on the target text information according to the target keyword and the target distribution information to obtain the target semantic type corresponding to the initial voice information includes: Obtaining an associated semantic type corresponding to the multi-socket socket, an initial keyword corresponding to the associated semantic type, and initial distribution information corresponding to the initial keyword; Performing similarity calculation based on the target keyword and the initial keyword to obtain a text similarity value and performing distribution similarity calculation based on the initial distribution information and the target distribution information to obtain a distribution similarity value; Determining a target correlation between the initial voice information and the associated semantic type according to the text similarity value and the distribution similarity value; The target semantic type corresponding to the initial speech information is obtained from the associated semantic type according to the target relevance.

6. The method according to claim 5, characterized in that The step of determining the target correlation between the initial speech information and the associated semantic type according to the text similarity value and the distribution similarity value comprises: Determine a fusion parameter according to the text similarity value and the distribution similarity value, and perform data fusion on the text similarity value and the distribution similarity value according to the fusion parameter to obtain a target similarity value; Obtaining a maximum similarity value corresponding to the target keyword from the target similarity value, and obtaining a word frequency weight corresponding to the target keyword from the initial distribution information; Determining the target relevance between the initial speech information and the associated semantic type according to the target similarity value and the word frequency weight; The target correlation is obtained according to the following formula: in, represents the target relevance between the initial voice information and the i-th associated semantic type; all represents the target number corresponding to the target keyword, represents the word frequency weight corresponding to the jth target keyword, num represents the target number corresponding to the initial keyword, represents the fusion parameter, represents the text similarity value between the j-th target keyword and the x-th initial keyword, represents the distribution similarity value between the x-th initial distribution information and the j-th target distribution information; exp represents an exponential function, and max represents obtaining the maximum value.

7. The method according to any one of claims 1 to 6, characterized in that Before collecting the initial voice information, the method further includes: Collecting wake-up voice information, and performing voice classification on the wake-up voice information to obtain a voice classification result corresponding to the wake-up voice information; When the voice classification result is a preset type, the voice collection module corresponding to the multi-jack socket is turned on, so that the initial voice information is collected according to the voice collection module.

8. The method according to claim 7, characterized in that After obtaining the target control result, the method further includes: Obtaining a target socket state corresponding to the multi-socket socket according to the target control result; The target socket state is sent to a user terminal communicatively connected to the multi-socket socket according to a communication module corresponding to the multi-socket socket, so that a target user obtains the target socket state corresponding to the multi-socket socket according to the user terminal.

9. A multi-socket voice control system based on natural language processing, characterized in that: include: A speech analysis module, used for collecting initial speech information, and classifying the initial speech information into language types to obtain relevant speech information corresponding to the initial speech information in a first language type; A speech clustering module is used to determine preset speech information, and perform speech clustering according to the preset speech information and the related speech information to obtain similar speech information corresponding to the related speech information, wherein the performing speech clustering according to the preset speech information and the related speech information to obtain similar speech information corresponding to the related speech information includes: performing discrete processing on the preset speech information to obtain first discrete data and performing discrete processing on the related speech information to obtain second discrete data; calculating the discrete distance between the first discrete data and the second discrete data according to the mutual correlation function; performing speech clustering on the preset speech information and the related speech information according to the discrete distance to obtain a target clustering result corresponding to the related speech information; obtaining an initial clustering cluster corresponding to the related speech information according to the target clustering result, and removing the related speech information from the initial clustering cluster to obtain a target clustering cluster; obtaining a target semantic type corresponding to each sub-speech information under the first language type in the target clustering cluster; performing data statistics on the target semantic type to obtain the maximum semantic type corresponding to the target clustering cluster; obtaining the similar speech information corresponding to the related speech information according to the maximum semantic type and the first language type; A mapping processing module, used for obtaining, from a voice mapping table, the ideal voice information corresponding to the relevant voice information in a second language type according to the similar voice information; A voice replacement module, used for performing voice replacement on the initial voice information according to the ideal voice information and the related voice information to obtain target voice information corresponding to the initial voice information; A speech recognition module, used to perform speech recognition on the target speech information according to a speech recognition model to obtain initial text information corresponding to the target speech information; A text correction module, used for performing text correction on the initial text information to obtain target text information corresponding to the target voice information; A data identification module, used to perform keyword identification on the target text information to obtain target keywords and target distribution information corresponding to the target keywords; A type classification module, used for semantically classifying the target text information according to the target keywords and the target distribution information to obtain a target semantic type corresponding to the initial voice information; A result control module is used to control the multi-jack socket according to the target semantic type to obtain the corresponding target control result under the initial voice information.

Images