Intelligent voice calling method and system

Abstract

The invention provides an intelligent voice calling method and system, and relates to the technical field of voice processing, and the method comprises the steps of voice receiving, instruction judgment, model construction, model training, number query, telephone dialing, dialing feedback and the like. The system comprises a voice receiving module, an instruction judgment module, a model construction module, a model training module, a number query module, a telephone dialing module and a dialing feedback module. Meanwhile, the translated texts of the single language components are preliminarily combined according to the voice, then the translated texts of the single language components are further combined into the phrase sequence conforming to the grammar rule through the phrases by means of the model, and the processing capacity of the cross-language instruction is improved through effective processing of the cross-language instruction.

Description

Technical Field

[0001] The present application relates to the field of voice processing technology, and in particular to an intelligent voice calling method and system. Background Art

[0002] With the continuous advancement of artificial intelligence technology, intelligent voice calling has been widely used in many scenarios such as telephone promotion and telephone consultation services. Intelligent voice calling can initiate outbound calls to users as the caller, or receive the user's call request as the called party, and conduct real-time conversations with users through intelligent voice. The emergence of intelligent voice calling methods not only greatly improves communication efficiency, but also provides users with a more convenient and efficient communication experience.

[0003] The Chinese invention patent with application publication number CN118488139A provides an intelligent voice call system, which performs corresponding tasks respectively through a connection module, a voice recognition module, a first translation module, a parsing module, a retrieval module, a second translation module, and a voice synthesis module that are sequentially connected in a closed loop, and at least one module simultaneously performs the tasks of its own module through multiple units, which can realize multi-language intelligent calls, and by performing tasks in the form of serial and parallel combination of each module, it can increase the speed of processing call-related data and the speed of answering calls, thereby improving call quality and user call experience.

[0004] The above technical solutions can realize multi-language intelligent calls, but they do not involve the ability to process cross-language instructions (i.e., two or more languages ​​in one sentence). Due to the significant differences in grammar, vocabulary, and expressions between different languages, it is often difficult for the system to accurately identify and understand such complex instructions. Once the system cannot correctly parse the user's cross-language instructions, it may lead to call interruption, misunderstanding, or execution error, which will seriously affect the user experience. Summary of the invention

[0005] In order to improve the processing capability of cross-language instructions, the present application provides an intelligent voice calling method and system.

[0006] In the first aspect, the present application provides an intelligent voice calling method, which adopts the following technical solution: An intelligent voice calling method, the method comprising: Voice reception: receive user's voice information in real time; Command judgment: judge whether the received voice message is a voice call command: If yes, then execute the model building steps; If not, then executing the step of voice receiving; Model construction: Based on building a cross-speech recognition model, including language recognition model and language combination model; Model training: including first processing, first sorting and second processing; First processing: using the speech information as input of the language recognition model to obtain first data; First sorting: sorting the first data according to the phrase arrangement order of the received voice information to obtain second data; Second processing: using the second data as input of the language combination model to obtain third data; Number query: based on the third data, search the contact list in the device to obtain the contact corresponding to the third data and the phone number corresponding to the contact; Phone call: dial the phone number corresponding to the contact to initiate a call; Dialing Feedback: During a call, the dialing feedback is announced.

[0007] By adopting the above technical solution, a cross-speech recognition model is constructed, including a language recognition model and a language combination model. The language recognition model is used to recognize and translate individual language components in the speech. At the same time, the translation texts of the individual language components are preliminarily combined according to the speech. Then, the language combination model is used to further combine the translation texts of the individual language components into a sequence of phrases that conform to grammatical rules. This helps the model to better understand the structure and meaning of cross-language instructions, improves the model's overall understanding of cross-language instructions, and improves the adaptability of intelligent voice call technology through effective processing of cross-language instructions. Moreover, based on the cross-speech recognition model, not only can mixed instructions containing two or more languages ​​be processed, but also flexible adjustments can be made according to the specific needs of users, so that the model can better recognize specific language combinations or terms in specific fields, and can respond to user needs more intelligently, thereby improving the user's overall call experience and satisfaction.

[0008] Optionally, after executing the step of receiving the voice and before executing the step of judging the instruction, the method further includes: First construction: Building a voice recognition model based on voiceprint recognition technology and sentiment analysis technology; First recognition: using the received voice information as the input of the sound recognition model to obtain the voice classification result; First judgment: judge whether the voice classification result is the preset authorized user: If yes, execute the second judgment; If not, then executing the step of voice receiving; Second judgment: Perform a fracture analysis on the received voice information to determine whether there is smooth fracture pronunciation: If yes, then execute the instruction judgment step; If not, the step of voice receiving is performed.

[0009] By adopting the above technical solution, based on voiceprint recognition technology, by extracting voiceprint features in the voice and comparing them with the pre-stored user voiceprint database, the user's identity can be quickly and accurately verified, preventing unauthorized users from invading the system by forging or imitating voice, thereby ensuring the security of the communication process. In order to further improve the accuracy of user identity recognition, voice fracture analysis technology is introduced. By performing fracture analysis on the user's voice and comparing the fracture features in the user's voice with the features in the pre-stored template, the user's identity can be further verified, which not only improves the accuracy of identity recognition, but also enhances the anti-spoofing ability, and ensures that only authorized users can access and operate, effectively preventing the risk of information leakage and malicious attacks, and providing users with a safer and more convenient communication experience.

[0010] Optionally, after executing the step of building the model and before executing the step of the first processing, the method further includes: Language acquisition: Get the language type set by the device and record it as the target language; First collection: collect a number of sentences containing mixed languages ​​and the corresponding translation texts in the target language, mark the start and end positions of each language part in each sentence, and mark the corresponding language tags, record each annotated voice data and the corresponding translation text as the first sample, collect the first samples, and obtain the first sample data set; First input: input the first sample into the language recognition model, divide the speech data and the corresponding translation text according to the start and end position marks of the speech data in the first sample, obtain a number of speech segments and corresponding translation text groups, and use the obtained number of speech segments and corresponding translation text groups as new first samples; First extraction: Use the shared layer of the language recognition model to extract the features of the first sample, which are recorded as shared features; Task branches: construct different numbers of language task branches based on different languages, and construct corresponding translation task branches based on different language branches and target languages; The first task is to input the shared features into each language task branch for task processing and obtain the task processing results of the language task branch; The second task: concatenate the task processing results of the language task branch and the shared features, and input the concatenated data into the corresponding translation task branch for task processing; First optimization: define a multi-task loss function, iteratively optimize the language recognition model by minimizing the task loss function, and use the optimized language recognition model as the new language recognition model.

[0011] By adopting the above technical solution, the start and end positions of each language part in each sentence of the sample data are marked and assigned corresponding language labels, so that the model can more accurately locate and identify different language components in subsequent tasks. At the same time, based on the multi-task learning framework, multiple language task branches and translation task branches are constructed, allowing the language recognition model to process multiple language translation tasks in parallel. While improving the processing ability of cross-language instructions, it can make full use of computing resources, shorten the processing time of the overall task, and provide users with a better communication and interaction experience.

[0012] Optionally, after executing the step of building the model and before executing the step of the first processing, the method further includes: Set window: set the size and overlap of the sliding window; Speech segmentation: Use a sliding window to segment the received speech information and use the segmented speech information as new speech information.

[0013] By adopting the above technical solution, by setting the size and overlap of the sliding window, it is helpful to flexibly process voice information of different lengths, and by segmenting the voice information based on the sliding window, the voice data can be accurately segmented into smaller, easier-to-process units, which helps the model to more accurately recognize each word or phrase in the voice, thereby improving the recognition accuracy of cross-language instructions. In addition, segmenting long voice information into shorter voice segments through the sliding window can reduce the computational burden of the model when processing each voice segment, which helps the model process cross-language instructions faster and improves the overall response speed.

[0014] Optionally, after executing the step of building the model and before executing the step of setting the window, the method further includes: Preset window: set the initial size of the sliding window; Feature calculation: calculate the energy change rate, spectrum change rate and signal-to-noise ratio of the current frame speech information; Adjusting the window: adjusting the size of the sliding window based on the energy change rate, the spectrum change rate and the signal-to-noise ratio, and using the adjusted sliding window as the pre-set sliding window size.

[0015] By adopting the above technical solution, the size of the sliding window is dynamically adjusted according to the characteristics of the current speech signal, including the energy change rate, spectrum change rate and signal-to-noise ratio of the current frame speech information, which helps to better capture important information in the speech signal and improve the adaptability and accuracy of the cross-language recognition model.

[0016] Optionally, after performing the first sorting step and before performing the second processing step, the method further includes: Second collection: collecting data of a parallel corpus, wherein the parallel corpus includes a source language and a corresponding original text; Word segmentation: Split each sentence of the original text into a word sequence of subject-verb-object structure, annotate the corresponding parts of speech for the split word sequence, and collect the annotated data to obtain a corpus text library; Second optimization: input the corpus text library into the language combination model for model inference, define the combination loss function, iteratively optimize the language combination model by minimizing the combination loss function, and use the optimized language combination model as the new language combination model.

[0017] By adopting the above technical solution, the original text is split into a word sequence of a subject-verb-object structure, so that the language combination model can understand the syntactic structure of the sentence more intuitively, which helps the language combination model to identify the core components in the sentence and their relationships, thereby optimizing the logic of language combination and making the generated text more fluent and coherent. Moreover, part-of-speech tagging of the split word sequence not only provides grammatical category information of the vocabulary, but also helps the language combination model to identify the functional role of the vocabulary in the sentence, which helps the language combination model to select vocabulary more accurately when combining languages, thereby improving the quality of the generated text.

[0018] Optionally, after executing the step of querying the number and before executing the step of dialing the phone, the method further includes: Third judgment: judging whether the number of contacts corresponding to the third data is 1: If yes, then execute the step of dialing the phone; If not, perform the steps confirmed by the personnel; Personnel announcement: Based on the found contacts and their corresponding phone numbers, the found contacts and phone numbers are announced in sequence, and the user is asked to confirm the contacts and phone numbers that need to be announced; Personnel determination: receiving the user's confirmation feedback and performing identification, based on the user's confirmation feedback, using the identified contact person and phone number as the contact person corresponding to the new third data and the phone number corresponding to the contact person, respectively, and executing the phone dialing step.

[0019] By adopting the above technical solution, when there are multiple targets in the query results, requesting user confirmation helps avoid wrong dialing due to misidentification or misoperation. At the same time, in cross-language instruction processing, due to language differences, there may be deviations in the understanding of instructions. Adding the above confirmation step as an error correction mechanism in cross-language instruction processing helps reduce operational errors caused by misunderstanding of instructions. Subsequent determination of the correct contact and phone number based on user feedback helps to improve the accuracy of cross-language instruction processing. In a cross-language environment, even if there is an misunderstanding of the initial cross-language instruction, the subsequent confirmation steps can accurately execute the user's intentions, enhance the user's interactive experience, and improve the accuracy of cross-language instruction processing.

[0020] Optionally, the method further comprises: Incoming call reception: receive incoming call number; Voice prompt: voice announcement based on the incoming call number received; User response: receiving the user's response information; The third process: inputting the user's response information into the cross-speech recognition model to obtain fourth data; Fourth judgment: judging whether the fourth data is a connection instruction: If yes, then answer the phone; If not, then execute the fifth judgment step; Fifth judgment: judging whether the fourth data includes a rejection instruction; If yes, then execute the sixth judgment step; If not, then execute the voice replay step; Sixth judgment: judging whether the fourth data includes a reply SMS instruction; If yes, edit the text message according to the fourth data and send it to the received caller number, and then hang up the call; If not: hang up the phone; Voice repeat: Repeat the received caller number.

[0021] By adopting the above technical solution and introducing a cross-speech recognition model, when receiving an incoming call, it is not limited to simple answer or reject instructions. It can also identify and process more complex reply SMS instructions based on the cross-speech recognition model, thereby achieving the purpose of providing personalized services based on the needs and preferences of different users, while simplifying the user operation process and improving user convenience.

[0022] In a second aspect, the present application provides an intelligent voice call system, wherein the segmentation system is applicable to the intelligent voice call method described in any one of the first aspects above, and the system includes: adopting the following technical solution: An intelligent voice calling system, the system comprising: Voice receiving module, used to receive user's voice information in real time; The command judgment module is used to judge whether the received voice information is a voice call command: If the received voice information is a voice call instruction, the model building module is triggered; If the received voice message is not a voice call instruction, the voice receiving module is triggered; A model building module, used to build a cross-speech recognition model, including a language recognition model and a language combination model; The model training module is communicatively connected with the model building module, the voice receiving module and the number query module, and includes a first processing unit, a first sorting unit and a second processing unit; A first processing unit, configured to use the received voice information as an input of a language recognition model to obtain first data; A first sorting unit, used to sort the first data according to the phrase arrangement order of the received voice information to obtain second data; A second processing unit, configured to use the second data as an input of the language combination model to obtain third data; A number query module, used to search the contact list in the device based on the third data, and obtain the contact corresponding to the third data and the telephone number corresponding to the contact; The phone dialing module is used to dial the phone number corresponding to the contact to initiate a call; The dialing feedback module is used to broadcast the dialing feedback during the call process.

[0023] By adopting the above technical solution, a cross-speech recognition model is built based on the model building module, including a language recognition model and a language combination model. The language recognition model is used to recognize and translate individual language components in the speech. At the same time, the translated texts of the individual language components are preliminarily combined according to the speech. Then, the language combination model is used to further combine the translated texts of the individual language components into a sequence of phrases that conform to grammatical rules. This helps the model to better understand the structure and meaning of cross-language instructions, improves the model's overall understanding ability of cross-language instructions, and through the effective processing of cross-language instructions, improves the adaptability of intelligent voice call technology, and improves the overall call experience and satisfaction of users.

[0024] In summary, the present application includes at least one of the following beneficial technical effects: 1. Construct a cross-speech recognition model, including a language recognition model and a language combination model. The language recognition model is used to recognize and translate individual language components in speech. At the same time, the translation texts of individual language components are preliminarily combined according to the speech. Then, the language combination model is used to further combine the translation texts of individual language components into a sequence of phrases that conform to grammatical rules, thereby improving the model's overall understanding of cross-language instructions. By effectively processing cross-language instructions, the adaptability of intelligent voice call technology is improved. Moreover, based on the cross-speech recognition model, not only can mixed instructions containing two or more languages ​​be processed, but also flexible adjustments can be made according to the specific needs of users, so that the model can better recognize specific language combinations or terms in specific fields, and can respond to user needs more intelligently, thereby improving the user's overall call experience and satisfaction.

[0025] 2. The start and end positions of each language part in each sentence of the sample data are marked and assigned corresponding language labels, so that the model can more accurately locate and identify different language components in subsequent tasks. At the same time, based on the multi-task learning framework, multiple language task branches and translation task branches are constructed, allowing the language recognition model to process multiple language translation tasks in parallel. While improving the processing ability of cross-language instructions, it can make full use of computing resources, shorten the processing time of the overall task, and provide users with a better communication and interaction experience.

[0026] 3. By setting the size and overlap of the sliding window, it helps to flexibly process voice information of different lengths. Moreover, by segmenting the voice information based on the sliding window, the voice data can be accurately segmented into smaller, easier-to-process units, which helps the model to more accurately recognize each word or phrase in the voice, thereby improving the recognition accuracy of cross-language instructions. In addition, segmenting long voice information into shorter voice segments through the sliding window can reduce the computational burden of the model when processing each voice segment, which helps the model process cross-language instructions faster and improves the overall response speed.

[0027] 4. The introduction of a cross-speech recognition model means that when receiving an incoming call, it is not limited to simple answer or reject commands. Instead, it can also identify and process more complex reply SMS commands based on the cross-speech recognition model, thereby achieving the purpose of providing personalized services based on the needs and preferences of different users. At the same time, it simplifies the user operation process and improves user convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of Example 1 of the present application; Figure 2 This is a flow chart of the S4 model training of Example 1 of the present application; Figure 3 This is a flowchart of the first training in S41 of Example 1 of the present application; Figure 4 This is a flowchart of the second training in S45 of Example 1 of the present application. DETAILED DESCRIPTION

[0029] The following combination Figures 1 to 4 This application is described in further detail.

[0030] Embodiment 1: This embodiment discloses an intelligent voice calling method, such as Figure 1As shown, the method includes: receiving the user's voice information in real time, determining whether the received voice information is a voice call instruction, and if not, continuing to receive the user's voice information; if so, based on building a cross-speech recognition model, including a language recognition model and a language combination model, using the voice information as the input of the language recognition model to obtain first data, sorting the first data according to the phrase arrangement order of the received voice information to obtain second data, using the second data as the input of the language combination model to obtain third data, based on the third data, searching the contact list in the device, obtaining the contact corresponding to the third data and the phone number corresponding to the contact, dialing the phone number corresponding to the contact, initiating a call, and broadcasting the phone call feedback during the call. This embodiment includes the following steps: S1 Voice Reception: Receive user's voice information in real time.

[0031] In this embodiment, the intelligent voice calling method is applicable to communication devices capable of making calls, and each communication device is provided with a microphone or other audio input device for receiving the user's voice information in real time.

[0032] In this embodiment, the received user's voice information is pre-processed by noise elimination, echo suppression, etc.

[0033] Noise elimination: Spectral subtraction, Wiener filtering or adaptive filters (such as LMS algorithm or RLS algorithm) can be used to remove background noise from voice information. Background noise includes traffic noise and crowd conversation.

[0034] Echo Suppression: Echo cancellation filters can be used to remove echo from voice messages caused by feedback from the device's audio equipment (such as microphones and speakers).

[0035] S2 command judgment: judge whether the received voice information is a voice call command.

[0036] If yes, then execute the model building step.

[0037] If not, the step of voice receiving is performed.

[0038] The voice call instruction content is pre-stored, and whether the received voice information is a voice call instruction can be determined by determining whether the received voice information contains a keyword of the voice call instruction.

[0039] S3 model construction: Based on building a cross-speech recognition model, including a language recognition model and a language combination model.

[0040] In this embodiment, the Transformer architecture or the Seq2Seq model can be used as the base model of the language recognition model. A part of the Transformer encoder layers are used as the shared layer of the language recognition model, and the remaining Transformer encoder layers and the fully connected layers that perform tasks in the Transformer model are used as the task layer of the language recognition model. The BERT model is used as the base model of the language combination model to construct a cross-speech recognition model.

[0041] S4 model training: It includes S41 first training, S42 window setting, S43 first processing, S44 first sorting, S45 second training, and S46 second processing, as Figure 2 shown.

[0042] S41 first training: It includes S411 language acquisition, S412 first collection, S413 first input, S414 first extraction, S415 task branching, S416 first task, S417 second task, and S418 first optimization, as Figure 3 shown.

[0043] S411 language acquisition: Obtain the language type set by the device and record it as the target language.

[0044] In this embodiment, obtain the current language configuration from the device settings. Assume that the language type configuration is "Mandarin Chinese", and record this language type as the target language, that is, record it as the target language (for example, Mandarin Chinese).

[0045] S412 first collection: Collect several sentences containing mixed languages and the corresponding translated texts in the target language. Mark the start and end positions of each language part in each sentence and mark the corresponding language labels. Record each marked speech data and the corresponding translated text as the first sample, and gather the first samples to obtain the first sample dataset.

[0046] For example, take "Call Han Meimei in the design department, call Han Meimei" as an example. Mark the part "Call Han Meimei in the design department" as Chinese, the start position of Chinese is "给", and the end position of Chinese is "话". Mark the part "call HanMeimei" as English, the start position of English is "c", and the end position of English is "i". Record the marked "Call Han Meimei in the design department, call Han Meimei" and the corresponding translated text "Call Han Meimei in the design department or call Han Meimei's phone" as the first sample.

[0047] S413 First input: Input the first sample into the language recognition model, divide the speech data and the corresponding translation text according to the start and end position marks of the speech data in the first sample, obtain a number of speech segments and corresponding translation text groups, and use the obtained several speech segments and corresponding translation text groups as new first samples.

[0048] S414 First extraction: Use the shared layer of the Transformer model to extract the features of the first sample, which are recorded as shared features.

[0049] The shared features include the word vector and position encoding features of the first sample.

[0050] S415 task branch: construct different numbers of language task branches based on different languages, and construct corresponding translation task branches based on different language branches and target languages.

[0051] Based on the existing language type data (e.g., English, Chinese, French, etc.), multiple task branches are constructed. Each branch specializes in processing the characteristics of a language. For each language, a corresponding number of translation task branches are constructed based on the target language (e.g., Mandarin Chinese). A multi-task learning framework is constructed, including language recognition task branches and translation task branches. The translation task branches correspond to the language task branches one by one. The translation task branch corresponding to each language task branch is used to translate the language into the target language.

[0052] S416 First Task: Input the shared features into each language task branch for task processing. Each language task branch will perform task processing based on the input shared features. For example, it will identify whether the input is English, Chinese or other languages, and obtain the task processing results of each task branch, indicating the language type involved in the input first sample.

[0053] S417 Second Task: Directly concatenate the task processing results and shared features of the language task branch, input the concatenated data into the translation task branch corresponding to the language task branch for task processing, and output the translated data.

[0054] S418 First Optimization: Define a multi-task loss function, iteratively optimize the language recognition model by minimizing the task loss function, and use the optimized language recognition model as a new language recognition model.

[0055] The multi-task loss function is defined as: .

[0056] Calculate the task loss of each task according to the prediction result and label of each task, substitute the task loss of each task into the above formula to get the multi-task loss value.

[0057] in, represents the multi-task loss function, is the cross entropy loss of the language task branch, is the cross entropy loss of the translation task branch, and are weight coefficients, .

[0058] In this embodiment, cross-validation or hold-out method can be used to evaluate the language recognition model in and The performance of different weight settings and choose the best performing one and Weighted combination.

[0059] Use a backpropagation algorithm (such as gradient descent or Adam optimizer) to update the weights of the shared layer and task branch, and iterate the optimization until the loss function converges or the training round is reached to obtain the optimized language recognition model, and use the optimized language recognition model as the new language recognition model.

[0060] S42 Window Settings: includes S421 preset window, S422 feature calculation, S423 adjustment window, S424 setting window and S425 speech segmentation.

[0061] S421 preset window: set the initial size of the sliding window; S422 Feature calculation: Calculate the energy change rate, spectrum change rate and signal-to-noise ratio of the current frame speech information.

[0062] The energy change rate is calculated based on the energy of the current frame of speech information and the energy of the previous frame of speech information. The short-time Fourier transform (STFT) can be used to calculate the spectrum of each frame of speech information, and the mean square error (MSE) or other distance measurement between the adjacent frame spectra is calculated as the spectrum change rate.

[0063] S423 adjusts the window: assigns a weight to the energy change rate, the spectrum change rate and the signal-to-noise ratio, and calculates the comprehensive characteristic value according to the comprehensive characteristic calculation formula.

[0064] The comprehensive feature calculation formula is: .

[0065] in, For comprehensive characteristics, is the energy change rate of the current frame speech information, is the frequency spectrum change rate of the current frame speech information, is the signal-to-noise ratio of the current frame speech information, , and are weights, .

[0066] in, , and The weight value can be determined by expert estimation method, hierarchical analysis method and other methods.

[0067] The size of the sliding window is adjusted according to the calculated comprehensive features, and the adjusted sliding window is used as the size of the pre-set sliding window.

[0068] The sliding window adjustment formula is: .

[0069] in, is the size of the sliding window after adjustment, is the initial size of the sliding window, is the adjustment factor, The value of can be determined by expert scoring method or analogy method.

[0070] S424 Setting window: setting the size and overlap of the sliding window; S425 Segmentation of voice: Use a sliding window to segment the received voice information, and use the segmented voice information as new voice information.

[0071] S43: First processing: using the voice information as the input of the language recognition model to obtain the first data.

[0072] S44 first sorting: The first data obtained is the translation text of each language. The translation text of each language is in word order, so the translation text of each language in the first data is sorted according to the phrase arrangement order of the received voice information, and the sorted phrases are recorded as second data.

[0073] S45 second training: including S451 second acquisition, S452 word segmentation processing and S453 second optimization, such as Figure 4 shown.

[0074] S451 Second collection: collecting data of a parallel corpus, where the parallel corpus includes a source language and a corresponding original text.

[0075] Collect parallel corpora containing the target language from various sources (such as books, articles, dialogues, film and television subtitles, etc.). The source language can be multiple languages ​​(such as Chinese, English, French, etc.), or it can be a collection of dialects from various regions of China.

[0076] S452 word segmentation processing: split each sentence of the original text into a word sequence of subject-verb-object structure, mark the corresponding parts of speech for the split word sequence, and collect the marked data to obtain a corpus text library.

[0077] S453 Second optimization: input the corpus text library into the language combination model for model reasoning, define the combination loss function, iteratively optimize the language combination model by minimizing the combination loss function, and use the optimized language combination model as the new language combination model.

[0078] S46 Second processing: using the second data as input of the language combination model to obtain third data.

[0079] S5 Number query: Based on the third data, search the contact list in the device to obtain the contact corresponding to the third data and the phone number corresponding to the contact; S6 Phone Call: Dial the phone number corresponding to the contact to initiate a call; S7 dialing feedback: During a call, the dialing feedback will be announced.

[0080] Embodiment 2: The difference from Embodiment 1 is as follows: After executing the step of receiving voice in S1 and before executing the step of judging the instruction in S2, the method further includes: S11 First Construction: Build a sound recognition model based on voiceprint recognition technology and sentiment analysis technology.

[0081] The voice data of the authorized user is collected, and the unique voiceprint features are extracted from the voice using voiceprint recognition algorithms (such as MFCC, Mel spectrogram, feature vector, etc.). Voiceprint feature extraction aims to capture individual voice characteristics (such as frequency, pitch, timbre, etc.). The voiceprint recognition model is trained using the extracted voiceprint features. In this embodiment, a convolutional neural network CNN or a long short-term memory network LSTM model based on deep learning can be used as the basic model of the voice recognition model.

[0082] S12 First Recognition: Using the received voice information as the input of the sound recognition model to obtain the voice classification result.

[0083] S13 first judgment: judging whether the voice classification result is a preset authorized user.

[0084] If yes, execute S14 second judgment.

[0085] If not, perform S1 voice reception.

[0086] S14 second judgment: perform a fracture analysis on the received voice information to determine whether there is a smooth fractured pronunciation.

[0087] If yes, execute S2 instruction to judge.

[0088] If not, perform S1 voice reception.

[0089] In this embodiment, a speech signal processing technology is used to detect whether there is a smooth pronunciation break based on short-time Fourier transform (STFT) or other signal analysis methods.

[0090] In this embodiment, based on voiceprint recognition technology, by extracting voiceprint features from speech and comparing them with the pre-stored user voiceprint database, the user's identity can be quickly and accurately verified, preventing unauthorized users from invading the system by forging or imitating voice, thereby ensuring the security of the communication process. In order to further improve the accuracy of user identity recognition, voice fracture analysis technology is introduced. By performing fracture analysis on the user's voice and comparing the fracture features in the user's voice with the features in the pre-stored template, the user's identity can be further verified, which not only improves the accuracy of identity recognition, but also enhances the anti-spoofing ability, and ensures that only authorized users can access and operate, effectively preventing the risk of information leakage and malicious attacks, and providing users with a safer and more convenient communication experience.

[0091] Embodiment 3: The difference from Embodiment 1 is as follows: After executing the step of S5 number query and before executing the step of S6 phone dialing, the method further includes: S51: Third determination: determining whether the number of contacts corresponding to the third data is 1: If so, execute S6 phone call; If not, execute S51 personnel confirmation; S52 Personnel broadcast: Based on the found contacts and corresponding phone numbers, the found contacts and phone numbers are broadcasted in sequence, and the user is confirmed with the contacts and phone numbers that need to be broadcasted; S53 personnel determination: receiving the user's confirmation feedback and performing identification, based on the user's confirmation feedback, using the identified contact person and phone number as the contact person corresponding to the new third data and the phone number corresponding to the contact person, respectively, and executing S6 phone dialing.

[0092] In this embodiment, when there are multiple targets in the query results, requesting user confirmation helps avoid incorrect dialing due to misidentification or misoperation. At the same time, in cross-language instruction processing, due to language differences, there may be deviations in the understanding of instructions. Adding the above confirmation step as an error correction mechanism in cross-language instruction processing helps reduce operational errors caused by misunderstanding of instructions. Subsequent determination of the correct contact and phone number based on user feedback helps to improve the accuracy of cross-language instruction processing. In a cross-language environment, even if there is an misunderstanding of the initial cross-language instruction, the user's intention can be accurately executed through subsequent confirmation steps, which enhances the user's interactive experience and improves the accuracy of cross-language instruction processing.

[0093] Embodiment 4: The difference from Embodiment 1 is as follows: The intelligent voice calling method further includes: S8 incoming call reception: Receive incoming call numbers.

[0094] S9 Voice Prompt: Voice announcement based on the incoming call number.

[0095] S10 User response: receiving user response information.

[0096] S101: The third process: inputting the user's response information into the cross-speech recognition model to obtain fourth data.

[0097] S102 Fourth determination: determining whether the fourth data is a connection instruction.

[0098] If yes, answer the call.

[0099] If not, the fifth determination S103 is performed.

[0100] S103 Fifth determination: determining whether the fourth data includes a rejection instruction.

[0101] If yes, the sixth determination S104 is performed.

[0102] If not, then perform voice replay S105.

[0103] S104 Sixth determination: determining whether the fourth data includes a reply SMS instruction.

[0104] If so, edit the text message according to the fourth data and send it to the received caller number, and then hang up the call.

[0105] If no: hang up the phone.

[0106] S105 Voice replay: Repeats the received caller number.

[0107] In this embodiment, relevant contents of the rejection instruction and the reply SMS instruction are pre-stored, and whether the rejection instruction or the reply SMS instruction is included is determined by judging the keywords of the rejection instruction or the reply SMS instruction.

[0108] In this embodiment, a cross-speech recognition model is introduced. When receiving an incoming call, it is not limited to simple answer or reject instructions. Instead, more complex reply SMS instructions can be identified and processed based on the cross-speech recognition model, thereby achieving the purpose of providing personalized services based on the needs and preferences of different users, while simplifying the user operation process and improving user convenience.

[0109] Embodiment 5: This embodiment discloses an intelligent voice calling system, the system comprising: The voice receiving module is used to receive the user's voice information in real time.

[0110] The command judgment module is used to judge whether the received voice information is a voice call command.

[0111] If the received voice information is a voice call instruction, the model building module is triggered.

[0112] If the received voice information is not a voice call instruction, the voice receiving module is triggered.

[0113] The model building module is used to build a cross-speech recognition model, including a language recognition model and a language combination model.

[0114] The model training module is communicatively connected with the model building module, the voice receiving module and the number query module, and includes a window setting unit, a first processing unit, a first sorting unit and a second processing unit.

[0115] The first processing unit is used to use the received voice information as input of the language recognition model to obtain first data.

[0116] The first sorting unit is used to sort the first data according to the phrase arrangement order of the received voice information to obtain the second data.

[0117] The second processing unit is used to use the second data as input of the language combination model to obtain third data.

[0118] The number query module is used to search the contact list in the device based on the third data to obtain the contact corresponding to the third data and the telephone number corresponding to the contact.

[0119] The phone dialing module is used to dial the phone number corresponding to the contact to initiate a call.

[0120] The dialing feedback module is used to broadcast the dialing feedback during the call process.

[0121] In this embodiment, a cross-speech recognition model is constructed based on a model building module, including a language recognition model and a language combination model. The language recognition model is used to recognize and translate individual language components in speech. At the same time, the translated texts of individual language components are preliminarily combined according to the speech. Then, the language combination model is used to further combine the translated texts of individual language components into a sequence of phrases that conform to grammatical rules. This helps the model to better understand the structure and meaning of cross-language instructions, improves the model's overall understanding ability of cross-language instructions, and through the effective processing of cross-language instructions, improves the adaptability of intelligent voice call technology, and improves the overall call experience and satisfaction of users.

[0122] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. An intelligent voice calling method, characterized in that: include: Voice reception: receive user's voice information in real time; Command judgment: judge whether the received voice message is a voice call command: If yes, then execute the model building steps; If not, then executing the step of voice receiving; Model construction: Based on building a cross-speech recognition model, including language recognition model and language combination model; Model training: including first processing, first sorting and second processing; First processing: using the speech information as input of the language recognition model to obtain first data; First sorting: sorting the first data according to the phrase arrangement order of the received voice information to obtain second data; Second processing: using the second data as input of the language combination model to obtain third data; Number query: based on the third data, search the contact list in the device to obtain the contact corresponding to the third data and the phone number corresponding to the contact; Phone call: dial the phone number corresponding to the contact to initiate a call; Dialing Feedback: During a call, the dialing feedback is announced.

2. The intelligent voice calling method according to claim 1, characterized in that: After executing the step of receiving the voice and before executing the step of judging the instruction, the method further includes: First construction: Building a voice recognition model based on voiceprint recognition technology and sentiment analysis technology; First recognition: using the received voice information as the input of the sound recognition model to obtain the voice classification result; First judgment: judge whether the voice classification result is the preset authorized user: If yes, execute the second judgment; If not, then executing the step of voice receiving; Second judgment: Perform a fracture analysis on the received voice information to determine whether there is smooth fracture pronunciation: If yes, then execute the instruction judgment step; If not, the step of voice receiving is performed.

3. The intelligent voice calling method according to claim 1, characterized in that: After executing the step of model building and before executing the step of first processing, the method further includes: Language acquisition: Get the language type set by the device and record it as the target language; First collection: collect a number of sentences containing mixed languages ​​and the corresponding translation texts in the target language, mark the start and end positions of each language part in each sentence, and mark the corresponding language tags, record each annotated voice data and the corresponding translation text as the first sample, collect the first samples, and obtain the first sample data set; First input: input the first sample into the language recognition model, divide the speech data and the corresponding translation text according to the start and end position marks of the speech data in the first sample, obtain a number of speech segments and corresponding translation text groups, and use the obtained number of speech segments and corresponding translation text groups as new first samples; First extraction: Use the shared layer of the language recognition model to extract the features of the first sample, which are recorded as shared features; Task branches: construct different numbers of language task branches based on different languages, and construct corresponding translation task branches based on different language branches and target languages; The first task is to input the shared features into each language task branch for task processing, and obtain the task processing results of the language task branch; The second task: concatenate the task processing results of the language task branch and the shared features, and input the concatenated data into the corresponding translation task branch for task processing; First optimization: define a multi-task loss function, iteratively optimize the language recognition model by minimizing the task loss function, and use the optimized language recognition model as the new language recognition model.

4. The intelligent voice calling method according to claim 1, characterized in that: After executing the step of model building and before executing the step of first processing, the method further includes: Set window: set the size and overlap of the sliding window; Speech segmentation: Use a sliding window to segment the received speech information and use the segmented speech information as new speech information.

5. The intelligent voice calling method according to claim 4, characterized in that: After the model building steps, but before the setup window steps, also include: Preset window: set the initial size of the sliding window; Feature calculation: calculate the energy change rate, spectrum change rate and signal-to-noise ratio of the current frame speech information; Adjusting the window: adjusting the size of the sliding window based on the energy change rate, the spectrum change rate and the signal-to-noise ratio, and using the adjusted sliding window as the pre-set sliding window size.

6. The intelligent voice calling method according to claim 1, characterized in that: After executing the first sorting step and before executing the second processing step, the method further includes: Second collection: collecting data of a parallel corpus, wherein the parallel corpus includes a source language and a corresponding original text; Word segmentation: Split each sentence of the original text into a word sequence of subject-verb-object structure, annotate the corresponding parts of speech for the split word sequence, and collect the annotated data to obtain a corpus text library; Second optimization: input the corpus text library into the language combination model for model inference, define the combination loss function, iteratively optimize the language combination model by minimizing the combination loss function, and use the optimized language combination model as the new language combination model.

7. The intelligent voice calling method according to claim 1, characterized in that: After executing the step of querying the number and before executing the step of dialing the phone, the method further includes: Third judgment: judging whether the number of contacts corresponding to the third data is 1: If yes, then execute the step of dialing the phone; If not, perform the steps confirmed by the personnel; Personnel announcement: Based on the found contacts and their corresponding phone numbers, the found contacts and phone numbers are announced in sequence, and the user is asked to confirm the contacts and phone numbers that need to be announced; Personnel determination: receiving the user's confirmation feedback and performing identification, based on the user's confirmation feedback, using the identified contact person and phone number as the contact person corresponding to the new third data and the phone number corresponding to the contact person, respectively, and executing the phone dialing step.

8. The intelligent voice calling method according to any one of claims 1 to 7, characterized in that: The method further comprises: Incoming call reception: receive incoming call number; Voice prompt: voice announcement based on the incoming call number received; User response: receiving the user's response information; The third process: inputting the user's response information into the cross-speech recognition model to obtain fourth data; Fourth judgment: judging whether the fourth data is a connection instruction: If yes, then answer the phone; If not, then execute the fifth judgment step; Fifth judgment: judging whether the fourth data includes a rejection instruction; If yes, then execute the sixth judgment step; If not, then execute the voice replay step; Sixth judgment: judging whether the fourth data includes a reply SMS instruction; If yes, edit the text message according to the fourth data and send it to the received caller number, and then hang up the call; If not: hang up the phone; Voice repeat: Repeat the received caller number.

9. An intelligent voice calling system, characterized in that: The system is applicable to the method according to any one of claims 1 to 8, and the system comprises: Voice receiving module, used to receive user's voice information in real time; The command judgment module is used to judge whether the received voice information is a voice call command: If the received voice information is a voice call instruction, the model building module is triggered; If the received voice message is not a voice call instruction, the voice receiving module is triggered; A model building module, used to build a cross-speech recognition model, including a language recognition model and a language combination model; The model training module is communicatively connected with the model building module, the voice receiving module and the number query module, and includes a first processing unit, a first sorting unit and a second processing unit; A first processing unit, configured to use the received voice information as an input of a language recognition model to obtain first data; A first sorting unit, used to sort the first data according to the phrase arrangement order of the received voice information to obtain second data; A second processing unit is used to use the second data as an input of the language combination model to obtain third data; A number query module, used to search the contact list in the device based on the third data, and obtain the contact corresponding to the third data and the telephone number corresponding to the contact; The phone dialing module is used to dial the phone number corresponding to the contact to initiate a call; The dialing feedback module is used to broadcast the dialing feedback during the call process.

Images