Real-time voice call translation device and call translation method

Abstract

The invention relates to the technical field of call translation, and particularly discloses a real-time voice call translation device and a call translation method, and the device comprises a sentence segmentation mark recognition module which is used for judging whether a sentence segmentation mark exists in newly received voice data in real time as a real-time sentence segmentation mark recognition result; the input statement obtaining module is used for obtaining an input statement text of a latest received input language based on a latest recognized sentence segmentation mark in the latest received voice data when the real-time sentence segmentation mark recognition result is that the sentence segmentation mark exists in the latest received voice data; the statement text translation module is used for translating the input statement text of the latest received input language based on the target translation language to obtain a real-time voice call translation result; the method and the device are used for timely and accurately translating latest received voice data.

Description

Technical Field

[0001] The present invention relates to the technical field of call translation, and in particular to a real-time voice call translation device and a call translation method. Background Art

[0002] At present, with the development of society, global exchanges in various fields are becoming more frequent. In the process of cross-border economic and trade, political exchanges, cultural communication, etc., the demand for communication between people of different language backgrounds is growing. Language barriers have become the biggest obstacle to communication. Therefore, the demand for real-time voice call translation is becoming increasingly urgent. The development of network communication technology enables voice data to be transmitted quickly, providing communication guarantee for real-time voice call translation. Whether through mobile networks or Wi-Fi, real-time transmission of voice data can be achieved. Speech recognition technology continues to advance and can convert speech into text, providing a basis for voice call translation. For example, through a neural network with speech recognition function, the speech recognition text of the speech to be translated can be obtained. There are already some translation software on the market that provide people with certain translation assistance in daily communication. These software provide reference and basis for the development of real-time voice call translation technology.

[0003] According to the application scenario of voice call translation, in order to ensure the accuracy and readability of the translation, it is generally necessary to generate the translation result of the sentence only after obtaining a complete sentence uttered by the interlocutor. Therefore, the recognition of sentence segmentation marks is the key to determining the accuracy and immediacy of voice call translation. However, in a continuous conversation, it is a major difficulty to recognize the sentence segmentation marks in the interlocutor's voice data.

[0004] Therefore, the present invention provides a real-time voice call translation device and a call translation method. Summary of the invention

[0005] The present invention provides a real-time voice call translation device and a call translation method, which are used to accurately identify and determine the sentence segmentation marks in the latest received voice data, so as to realize timely and accurate translation of the latest received voice data.

[0006] The present invention provides a real-time voice call translation device, comprising:

[0007] A sentence segmentation mark recognition module is used to determine in real time whether there is a sentence segmentation mark in the latest received voice data as a real-time sentence segmentation mark recognition result;

[0008] An input sentence acquisition module, for acquiring the input sentence text of the latest received input language based on the latest recognized sentence segmentation mark in the latest received voice data when the real-time sentence segmentation mark recognition result indicates that the latest received voice data contains a sentence segmentation mark;

[0009] The sentence text translation module is used to determine the target translation language according to the international area code of the incoming / outgoing call, and translate the input sentence text of the latest received input language based on the target translation language to obtain real-time voice call translation results.

[0010] Preferably, the sentence segmentation mark recognition module includes:

[0011] The text translation submodule is used to convert the real-time received voice data into a text data form containing pause marks to obtain real-time voice text data of the input language;

[0012] The sentence segmentation mark recognition submodule is used to recognize whether there is a sentence segmentation mark in the real-time speech text data of the input language as a real-time sentence segmentation mark recognition result.

[0013] Preferably, the text translation submodule includes:

[0014] A text translation unit, used to convert the voice data received in real time into a text data form to obtain the initial voice text data of the input language;

[0015] A pause mark recognition unit, used for recognizing all pause marks in the voice data received in real time;

[0016] The mark marking unit is used to mark all pause marks in the initial voice text data of the input language in real time to obtain the real-time voice text data of the input language.

[0017] Preferably, the pause mark recognition unit comprises:

[0018] The curve fitting subunit is used to fit the short-time energy curve of the input voice signal in real time based on the short-time energy value of the voice signal of the voice data received in real time in each preset short time period;

[0019] An interval recognition subunit is used to recognize in real time all short-time energy abnormal curve intervals in the short-time energy curve of the input speech signal;

[0020] The endpoint time determination subunit is used to determine the pause mark start time and the pause mark end time of all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal;

[0021] A pause mark determination subunit is used to determine all pause marks in the voice data received in real time based on the pause mark start time and the pause mark end time of all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal;

[0022] Among them, all abnormal short time periods of each short-time energy abnormal curve interval include a left abnormal short time period and a right abnormal short time period of each short-time energy abnormal curve interval.

[0023] Preferably, the interval identification subunit includes:

[0024] A first mean value calculation terminal is used to determine all energy sudden drop curve segments in the short-time energy curve of the input speech signal, and calculate the average short-time energy value of each energy sudden drop curve segment;

[0025] The second mean value calculation terminal is used to calculate the average short-time energy value of all parts of the short-time energy curve of the input voice signal except all energy sudden drop curve segments in the short-time energy curve of the input voice signal as a reference average short-time energy value;

[0026] The sag ratio calculation terminal is used to calculate the ratio of the average short-time energy value of each energy sag curve segment to the reference average short-time energy value as the average energy sag ratio of each energy sag curve segment;

[0027] The curve segment screening end is used to treat all energy sudden drop curve segments whose average energy sudden drop ratio is less than a preset threshold value as all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal.

[0028] Preferably, the endpoint time determination subunit includes:

[0029] The endpoint period determination terminal is used to regard the preset short period corresponding to the short-time energy value at the left endpoint of each short-time energy abnormality curve interval as the left abnormal short period, and at the same time, regard the preset short period corresponding to the short-time energy value at the right endpoint of the short-time energy abnormality curve interval as the right abnormal short period;

[0030] The endpoint time determination end is used to identify the start time of the pause mark in the left abnormal short time period of each short-time energy abnormal curve interval based on the left neighboring voice signal segments and the right neighboring voice signal segments of all abnormal short time periods of each short-time energy abnormal curve interval, and at the same time, identify the end time of the pause mark in the right abnormal short time period of each short-time energy abnormal curve interval.

[0031] Preferably, the endpoint moment determination end includes:

[0032] The time period merging sub-terminal is used to merge each abnormal short time period of the short-time energy abnormal curve interval with the time period corresponding to the corresponding left neighboring speech signal segment and the time period corresponding to the right neighboring speech signal segment to obtain a reference time period of each short-time energy abnormal curve interval;

[0033] The signal segment division sub-end is used to determine a plurality of sampling points in each abnormal short time period of each short-time energy abnormal curve interval at a preset interval, and based on each sampling point, divide the voice signal of the part of the voice data corresponding to the reference time period of the short-time energy abnormal curve interval into a left reference voice signal segment and a right reference voice signal segment corresponding to the sampling point;

[0034] The zero-crossing rate difference calculating sub-terminal is used to take the difference between the zero-crossing rate of the left reference speech signal segment and the zero-crossing rate of the right reference speech signal segment at each sampling point as the left-right zero-crossing rate difference at each sampling point;

[0035] The time determination sub-terminal is used to take the sampling point with the largest left and right zero-crossing rate difference among all the sampling points in the left abnormal short time period of each short-time energy abnormal curve interval as the starting time of the pause mark, and at the same time, take the sampling point with the smallest left and right zero-crossing rate difference among all the sampling points in the right abnormal short time period of each short-time energy abnormal curve interval as the starting time of the pause mark.

[0036] Preferably, the sentence segmentation mark recognition submodule includes:

[0037] A word segmentation unit, used to input the real-time speech text data of the input language into the word segmentation model to obtain the word segmentation result of the real-time speech text data of the input language;

[0038] A duration determination unit, used to determine the duration of all pause marks in the real-time speech text data of the input language;

[0039] The sentence segmentation mark recognition and judgment unit is used to judge whether there is a sentence segmentation mark in the real-time speech text data of the input language as a real-time sentence segmentation mark recognition result based on the part of speech and meaning of each word contained in the word segmentation result and the duration of all pause marks.

[0040] Preferably, the sentence segmentation mark recognition and judgment unit includes:

[0041] A model acquisition subunit is used to acquire a sentence segmentation mark recognition model;

[0042] The sentence segmentation mark judgment subunit is used to input the parts of speech and meanings of all words contained in the word segmentation results and the duration of all pause marks into the sentence segmentation mark recognition model, and determine whether there are sentence segmentation marks in the real-time speech text data of the input language as the real-time sentence segmentation mark recognition result.

[0043] The present invention provides a real-time voice call translation method, which is applied to any of the above real-time voice call translation devices, comprising:

[0044] S1: Real-time determination of whether there is a sentence segmentation mark in the latest received voice data as a real-time sentence segmentation mark recognition result;

[0045] S2: when the real-time segmentation mark recognition result indicates that the segmentation mark exists in the latest received voice data, based on the segmentation mark newly recognized in the latest received voice data, obtaining the input sentence text of the latest received input language;

[0046] S3: Determine a target translation language according to the international area code of the incoming / outgoing call, translate the input sentence text of the latest received input language based on the target translation language, and obtain a real-time voice call translation result.

[0047] The beneficial effect of the present invention compared with the prior art is: by accurately identifying and judging the sentence segmentation marks in the latest received voice data, timely and accurate translation of the latest received voice data is achieved.

[0048] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0049] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0051] Figure 1 Schematic diagram of internal functional modules of a real-time voice call translation device in an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of internal functional submodules of a sentence segmentation mark recognition module in an embodiment of the present invention;

[0053] Figure 3 The figure is a flow chart of a real-time voice call translation method in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0055] Embodiment 1:

[0056] The present invention provides a real-time voice call translation device, referring to Figure 1 ,include:

[0057] A sentence segmentation mark recognition module is used to determine in real time whether there is a sentence segmentation mark in the latest received voice data as a real-time sentence segmentation mark recognition result;

[0058] An input sentence acquisition module, for acquiring the input sentence text of the latest received input language based on the latest recognized sentence segmentation mark in the latest received voice data when the real-time sentence segmentation mark recognition result indicates that the latest received voice data contains a sentence segmentation mark;

[0059] The sentence text translation module is used to determine the target translation language according to the international area code of the incoming / outgoing call, and translate the input sentence text of the latest received input language based on the target translation language to obtain real-time voice call translation results.

[0060] In this embodiment, the voice data is the voice data of the party to be translated obtained during the real-time voice call.

[0061] In this embodiment, the sentence break mark is a mark between adjacent sentences in the speech data, and its expression includes: the pause duration of the speech signal and the interval duration between adjacent sentences determined according to the meaning of the speech text.

[0062] In this embodiment, the real-time segmentation mark recognition result includes two situations: the segmentation mark exists in the latest received voice data and the segmentation mark does not exist in the latest received voice data.

[0063] In this embodiment, based on the most recently recognized sentence segmentation mark in the most recently received voice data, the most recently received input sentence text of the input language is obtained, that is:

[0064] The content from the last recognized segmentation mark in the most recently received voice data to the most recently recognized segmentation mark is regarded as the input sentence text of the most recently received input language.

[0065] In this embodiment, the input sentence text of the most recently received input language is translated based on the target translation language to obtain a real-time voice call translation result, namely:

[0066] The input sentence text of the most recently received input language is translated into the target translation language to obtain a real-time voice call translation result.

[0067] In this embodiment, the real-time voice call translation device includes an embedded computer module, a Bluetooth module, a WiFi module, a power supply and storage module, and a text-to-speech software (which includes the above sentence segmentation mark recognition module, input sentence acquisition module, and sentence text translation module);

[0068] The implementation principle and process of the device of the present invention are as follows:

[0069] 1. Call preparation:

[0070] Need to use with supporting mobile phone software;

[0071] The supporting mobile phone software needs to enable the mobile phone system dialing permission, call record permission and microphone permission;

[0072] When the user is on a call, the device should be connected to the mobile phone software via Bluetooth;

[0073] When users talk in different languages, one party starts the software and pre-selects the other party's language before dialing or answering the call.

[0074] 2. During a cross-language voice call:

[0075] The software monitors the voice input from the microphone and the voice information of the other party in the voice call network at the same time, and packages them separately and sends them to the translation device via Bluetooth;

[0076] The device receives voice files transmitted by the software via Bluetooth, and converts voice to text through built-in software, automatically calling the corresponding offline / online translation software API according to the WiFi connection status;

[0077] Through offline / online text translation software, text files can be translated between the languages ​​of both parties on the call;

[0078] The translated text file is converted into a voice file through text-to-speech software and transmitted back to the mobile phone software via Bluetooth;

[0079] The mobile phone software plays the corresponding file through the microphone or uploads it to the voice call network.

[0080] The beneficial effects of the above technology are as follows: the sentence segmentation mark recognition module can judge the sentence segmentation marks in the voice data in real time, thereby improving the accuracy and timeliness of voice processing. The input sentence acquisition module can obtain the input sentence text according to the latest recognized sentence segmentation mark, thereby ensuring the integrity and accuracy of the input sentence. The sentence text translation module can translate the latest input sentence text into the target language, quickly obtain the real-time voice call translation result, and improve the efficiency and practicality of the translation. The overall solution helps to achieve smooth and accurate real-time voice call translation and meet the needs of cross-language communication. It can effectively avoid translation errors caused by improper sentence segmentation or inaccurate sentence acquisition, thereby improving the quality of translation.

[0081] Embodiment 2:

[0082] Based on Example 1, the sentence segmentation mark recognition module refers to Figure 2 ,include:

[0083] The text translation submodule is used to convert the real-time received voice data into a text data form containing pause marks to obtain real-time voice text data of the input language;

[0084] The sentence segmentation mark recognition submodule is used to recognize whether there is a sentence segmentation mark in the real-time speech text data of the input language as a real-time sentence segmentation mark recognition result.

[0085] In this embodiment, the input language is the language of the voice data currently to be translated.

[0086] In this embodiment, the pause mark refers to the duration of a pause that occurs in the speech signal.

[0087] The beneficial effects of the above technology are as follows: the text translation submodule can convert speech data into text data containing pause marks, providing a clearer and more accurate basis for subsequent processing. It helps to improve the quality and readability of speech data converted into text. The sentence mark recognition submodule can accurately determine the sentence mark in real-time speech text data, improving the accuracy of subsequent processing. It provides strong support for efficient and accurate speech processing and related applications. It makes the entire speech processing process more standardized and intelligent.

[0088] Embodiment 3:

[0089] Based on Example 2, the text translation submodule includes:

[0090] A text translation unit, used to convert the voice data received in real time into a text data form to obtain the initial voice text data of the input language;

[0091] A pause mark recognition unit, used for recognizing all pause marks in the voice data received in real time;

[0092] The mark marking unit is used to mark all pause marks in the initial voice text data of the input language in real time to obtain the real-time voice text data of the input language.

[0093] The beneficial effects of the above technology are as follows: the text translation unit can convert real-time voice data into text data, laying the foundation for subsequent processing and improving the convenience of data processing. The pause mark recognition unit can recognize the pause marks in the voice in real time, which helps to grasp the rhythm and semantics of the voice more accurately. The mark marking unit marks the pause marks in the text data in real time, making the text more consistent with the characteristics of the original voice and enhancing the expressiveness and accuracy of the text. The overall process can improve the quality and accuracy of speech-to-text conversion and provide more reliable data support for related applications. It helps to achieve a more natural and more practical speech-to-text processing effect.

[0094] Embodiment 4:

[0095] Based on Embodiment 3, the pause mark recognition unit includes:

[0096] The curve fitting subunit is used to fit the short-time energy curve of the input voice signal in real time based on the short-time energy value of the voice signal of the voice data received in real time in each preset short time period;

[0097] An interval recognition subunit is used to recognize in real time all short-time energy abnormal curve intervals in the short-time energy curve of the input speech signal;

[0098] The endpoint time determination subunit is used to determine the pause mark start time and the pause mark end time of all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal;

[0099] A pause mark determination subunit is used to determine all pause marks in the voice data received in real time based on the pause mark start time and the pause mark end time of all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal;

[0100] Among them, all abnormal short time periods of each short-time energy abnormal curve interval include a left abnormal short time period and a right abnormal short time period of each short-time energy abnormal curve interval.

[0101] In this embodiment, the preset short period of time is a preset shorter time period, such as twenty milliseconds.

[0102] In this embodiment, the short-time energy value is a parameter used to describe the energy of a speech signal in a short time.

[0103] In this embodiment, the short-time energy curve of the input speech signal is a curve obtained by connecting and fitting the short-time energy values ​​of all preset short time periods in time sequence.

[0104] In this embodiment, the short-time energy abnormal curve interval is a voice signal of a portion of voice data within a continuous period in which the short-time energy of the input voice signal short-time energy curve begins to drop sharply and the short-time energy is stable at the value after the sudden drop.

[0105] In this embodiment, the pause mark starting time is the starting time of the pause mark.

[0106] In this embodiment, the pause mark termination time is the termination time of the pause mark.

[0107] In this embodiment, based on the pause mark start time and the pause mark end time of all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal, all the pause marks in the voice data received in real time are determined, that is:

[0108] The period from the start time of the pause mark of all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal to the end time of the pause mark is regarded as all the pause marks of the voice data received in real time.

[0109] The beneficial effects of the above technologies are as follows: the curve fitting subunit can more intuitively display the energy change trend of the voice signal by fitting the short-time energy curve of the input voice signal. The interval recognition subunit can identify the short-time energy abnormal curve interval in real time, which helps to accurately locate the area where pauses may exist. The endpoint moment determination subunit can determine the start and end moments of the abnormal curve interval, providing key time points for accurately judging pauses. The pause mark determination subunit determines the pause mark based on these moments, which improves the accuracy and reliability of pause judgment. The overall solution can effectively and accurately identify pause marks from complex voice data, providing strong support for subsequent operations of voice processing.

[0110] Embodiment 5:

[0111] Based on Example 4, the interval identification subunit includes:

[0112] A first mean value calculation terminal is used to determine all energy sudden drop curve segments in the short-time energy curve of the input speech signal, and calculate the average short-time energy value of each energy sudden drop curve segment;

[0113] The second mean value calculation terminal is used to calculate the average short-time energy value of all parts of the short-time energy curve of the input voice signal except all energy sudden drop curve segments in the short-time energy curve of the input voice signal as a reference average short-time energy value;

[0114] The sag ratio calculation terminal is used to calculate the ratio of the average short-time energy value of each energy sag curve segment to the reference average short-time energy value as the average energy sag ratio of each energy sag curve segment;

[0115] The curve segment screening end is used to treat all energy sudden drop curve segments whose average energy sudden drop ratio is less than a preset threshold value as all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal.

[0116] In this embodiment, all energy drop curve segments are determined in the short-time energy curve of the input speech signal, including:

[0117] Identify all energy drop points in the short-term energy curve of the input speech signal;

[0118] And in the short-time energy curve of the input speech signal, the longest time period with each energy drop point as the starting point, in which the ratio of the absolute value of the difference between the maximum short-time energy value and the minimum short-time energy value and the difference between the short-time energy value of the corresponding energy drop point and the short-time energy value of the adjacent previous sampling point of the energy drop point is less than a preset ratio (for example, one tenth) is determined as the energy drop curve segment.

[0119] In this embodiment, the average short-time energy value of the energy sudden drop curve segment is the average value of the short-time energy values ​​of all sampling points in the energy sudden drop curve segment.

[0120] In this embodiment, the average short-time energy value of the short-time energy curve of all parts of the input voice signal except all energy sudden drop curve segments in the short-time energy curve of the input voice signal is calculated, that is, the average value of the short-time energy values ​​of all sampling points in the short-time energy curve of all parts of the input voice signal except all energy sudden drop curve segments in the short-time energy curve of the input voice signal.

[0121] In this embodiment, the preset threshold is a threshold of the average energy drop ratio prepared in advance for screening the short-term energy abnormality curve interval.

[0122] The beneficial effects of the above technology are as follows: the first mean calculation end can determine the energy drop curve segment and calculate its average short-time energy value, providing basic data for subsequent analysis. The second mean calculation end calculates the average short-time energy value excluding the energy drop curve segment as a reference, making the analysis more comprehensive and comparative. The drop ratio calculation end obtains the average energy drop ratio of each energy drop curve segment, which helps to quantify the degree of energy change. The curve segment screening end screens out the short-time energy abnormal curve interval based on the preset threshold, which improves the accuracy and scientificity of the abnormal interval judgment. The overall solution can accurately identify abnormal intervals from the short-time energy curve of the input speech signal, providing strong support for the optimization and improvement of speech processing.

[0123] Embodiment 6:

[0124] Based on Embodiment 4, the endpoint time determination subunit includes:

[0125] The endpoint period determination terminal is used to regard the preset short period corresponding to the short-time energy value at the left endpoint of each short-time energy abnormality curve interval as the left abnormal short period, and at the same time, regard the preset short period corresponding to the short-time energy value at the right endpoint of the short-time energy abnormality curve interval as the right abnormal short period;

[0126] The endpoint time determination end is used to identify the start time of the pause mark in the left abnormal short time period of each short-time energy abnormal curve interval based on the left neighboring voice signal segments and the right neighboring voice signal segments of all abnormal short time periods of each short-time energy abnormal curve interval, and at the same time, identify the end time of the pause mark in the right abnormal short time period of each short-time energy abnormal curve interval.

[0127] In this embodiment, the preset short period corresponding to the short-time energy value at the left endpoint of the short-time energy anomaly curve interval is the preset short period corresponding to the short-time energy value corresponding to the leftmost sampling point in the short-time energy anomaly curve interval.

[0128] In this embodiment, the preset short period corresponding to the short-time energy value at the right endpoint of the short-time energy anomaly curve interval is the preset short period corresponding to the short-time energy value corresponding to the rightmost sampling point in the short-time energy anomaly curve interval.

[0129] In this embodiment, the left neighboring speech signal segment and the right neighboring speech signal segment of the abnormally short time period are speech signal segments of preset duration adjacent to the abnormally short time period and located on the left and right sides of the abnormally short time period, respectively, in the speech signals of the speech data received in real time.

[0130] The beneficial effects of the above technology are as follows: the endpoint time period determination end can accurately define the short time period corresponding to the short-time energy values ​​of the left and right endpoints of the short-time energy anomaly curve interval, providing a clear range for the subsequent determination of the pause time. It helps to locate the abnormal short time period more accurately and improve the accuracy of pause mark recognition. The endpoint time determination end identifies the start and end times of the pause mark by analyzing the neighborhood voice signal segments of the abnormal short time period, making the pause judgment more reliable. It can effectively and accurately find the start and end times of the pause from the complex short-time energy anomaly curve, improving the accuracy and quality of voice processing. For voice-related applications, such as voice recognition and translation, it provides more accurate pause information and improves user experience and application effects.

[0131] Embodiment 7:

[0132] Based on Embodiment 6, the endpoint time determination includes:

[0133] The time period merging sub-terminal is used to merge each abnormal short time period of the short-time energy abnormal curve interval with the time period corresponding to the corresponding left neighboring speech signal segment and the time period corresponding to the right neighboring speech signal segment to obtain a reference time period of each short-time energy abnormal curve interval;

[0134] The signal segment division sub-end is used to determine a plurality of sampling points in each abnormal short time period of each short-time energy abnormal curve interval at a preset interval, and based on each sampling point, divide the voice signal of the part of the voice data corresponding to the reference time period of the short-time energy abnormal curve interval into a left reference voice signal segment and a right reference voice signal segment corresponding to the sampling point;

[0135] The zero-crossing rate difference calculating sub-terminal is used to take the difference between the zero-crossing rate of the left reference speech signal segment and the zero-crossing rate of the right reference speech signal segment at each sampling point as the left-right zero-crossing rate difference at each sampling point;

[0136] The time determination sub-terminal is used to take the sampling point with the largest left and right zero-crossing rate difference among all the sampling points in the left abnormal short time period of each short-time energy abnormal curve interval as the starting time of the pause mark, and at the same time, take the sampling point with the smallest left and right zero-crossing rate difference among all the sampling points in the right abnormal short time period of each short-time energy abnormal curve interval as the starting time of the pause mark.

[0137] In this embodiment, each abnormal short time period of the short-time energy abnormal curve interval is combined with the time period corresponding to the corresponding left neighboring speech signal segment and the time period corresponding to the right neighboring speech signal segment, that is:

[0138] The time period corresponding to the left neighboring speech signal segment corresponding to each abnormal short time period in the short-time energy abnormality curve interval, the abnormal short time period, and the time period corresponding to the corresponding right neighboring speech signal segment are sequentially connected.

[0139] In this embodiment, the preset interval is a time interval prepared in advance for determining the sampling points, for example, 20 milliseconds.

[0140] In this embodiment, the speech signal of the partial speech data corresponding to the reference time period of the short-time energy anomaly curve interval is divided into a left reference speech signal segment and a right reference speech signal segment corresponding to the sampling point, that is, the speech signal segment located on the left side of the sampling point in the speech signal of the partial speech data corresponding to the reference time period of the short-time energy anomaly curve interval is taken as the left reference speech signal segment, and the speech signal segment located on the right side of the sampling point is taken as the right reference speech signal segment.

[0141] In this embodiment, the zero-crossing rate of the left (right) reference speech signal segment refers to the number of times the speech signal crosses the zero value per unit time, and its value is the ratio of the number of times the left (right) reference speech signal segment crosses the zero value to the duration of the left (right) reference speech signal segment.

[0142] The beneficial effects of the above technologies are as follows: the time period merging sub-end provides a richer data basis for a more comprehensive analysis of pauses by merging abnormally short time periods and their neighboring speech signal segments. The signal segment division sub-end achieves fine processing of speech data and improves the accuracy of analysis by determining sampling points and dividing speech signal segments. The zero crossing rate difference sub-end calculates the left and right zero crossing rate differences, providing an effective quantitative indicator for determining the pause moment. The moment determination sub-end can accurately find the start and end moments of the pause mark, greatly improving the accuracy and reliability of pause judgment. The overall solution can accurately identify the moment of the pause mark from a complex speech signal, which helps to improve the quality and effect of speech processing.

[0143] Embodiment 8:

[0144] On the basis of Example 2, the sentence segmentation mark recognition submodule includes:

[0145] A word segmentation unit, used to input the real-time speech text data of the input language into the word segmentation model to obtain the word segmentation result of the real-time speech text data of the input language;

[0146] A duration determination unit, used to determine the duration of all pause marks in the real-time speech text data of the input language;

[0147] The sentence segmentation mark recognition and judgment unit is used to judge whether there is a sentence segmentation mark in the real-time speech text data of the input language as a real-time sentence segmentation mark recognition result based on the part of speech and meaning of each word contained in the word segmentation result and the duration of all pause marks.

[0148] In this embodiment, the word segmentation model is trained in advance using a large amount of text data in an input language and the analysis results obtained after manually performing word segmentation operations on the aforementioned text data as training samples. During the training process, the text data is used as the model input, and the word segmentation results are used as the model output.

[0149] In this embodiment, the word segmentation result includes the result of dividing the real-time voice text data into a plurality of words.

[0150] The beneficial effects of the above technology are as follows: the word segmentation unit uses the word segmentation model to segment the real-time speech text data, providing a more detailed and accurate basis for subsequent analysis and processing. The duration determination unit can determine the duration of the pause mark, providing important time dimension information for judging whether it is a sentence segmentation mark. The sentence segmentation mark recognition and judgment unit comprehensively considers the part of speech, meaning and pause duration, and improves the accuracy and rationality of sentence segmentation mark recognition. It helps to understand and process the input speech text more accurately, and improve the quality and effect of speech text processing. It provides more accurate sentence segmentation judgment for related speech applications, such as speech translation, speech to text, etc., and enhances the practicality and reliability of the application.

[0151] Embodiment 9:

[0152] On the basis of Embodiment 8, the sentence segmentation mark recognition and judgment unit includes:

[0153] A model acquisition subunit is used to acquire a sentence segmentation mark recognition model;

[0154] The sentence segmentation mark judgment subunit is used to input the parts of speech and meanings of all words contained in the word segmentation results and the duration of all pause marks into the sentence segmentation mark recognition model, and determine whether there are sentence segmentation marks in the real-time speech text data of the input language as the real-time sentence segmentation mark recognition result.

[0155] In this embodiment, the sentence segmentation mark recognition model is pre-trained using the parts of speech and meanings of all words and the duration of all pause marks contained in the word segmentation results of a large number of real-time speech text data of the input language, as well as the results of manually judging whether there are sentence segmentation marks in the real-time speech text data of the input language based on the above features as training samples;

[0156] During the training process, the word segmentation results of the real-time speech text data of the input language, including the parts of speech and meanings of all words and the duration of all pause marks, are the model input, while the manual judgment result of whether there are sentence break marks in the real-time speech text data of the input language based on the above features is the model output.

[0157] The beneficial effects of the above technology are as follows: the model acquisition subunit acquires the sentence mark recognition model, which provides an effective tool and basis for accurate judgment. The sentence mark judgment subunit improves the scientificity and accuracy of the judgment by inputting multi-dimensional information such as part of speech, meaning and pause duration into the model for judgment. With the help of a special model, the subjectivity and error of manual judgment can be reduced, and the reliability of sentence mark recognition can be improved. It helps to process a large amount of speech text data more efficiently and improve processing speed and efficiency. It can adapt to the characteristics of different languages ​​and contexts, and enhance the versatility and applicability of sentence mark recognition.

[0158] Embodiment 10:

[0159] The present invention provides a real-time voice call translation method, which is applied to the real-time voice call translation device described in any one of embodiments 1 to 9, referring to Figure 3 ,include:

[0160] S1: Real-time determination of whether there is a sentence segmentation mark in the latest received voice data as a real-time sentence segmentation mark recognition result;

[0161] S2: when the real-time segmentation mark recognition result indicates that the segmentation mark exists in the latest received voice data, based on the segmentation mark newly recognized in the latest received voice data, obtaining the input sentence text of the latest received input language;

[0162] S3: Determine a target translation language according to the international area code of the incoming / outgoing call, translate the input sentence text of the latest received input language based on the target translation language, and obtain a real-time voice call translation result.

[0163] The beneficial effects of the above technology are as follows: S1 can determine the punctuation marks in the voice data in real time, thereby improving the accuracy and timeliness of voice processing. S2 can obtain the input sentence text based on the latest identified punctuation marks, thereby ensuring the completeness and accuracy of the input sentence. S3 can translate the latest input sentence text into the target language, quickly obtain real-time voice call translation results, and improve the efficiency and practicality of the translation. The overall solution helps to achieve smooth and accurate real-time voice call translation and meet the needs of cross-language communication. It can effectively avoid translation errors caused by improper punctuation or inaccurate sentence acquisition, and improve translation quality.

[0164] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A real-time voice call translation device, characterized in that: include: A sentence segmentation mark recognition module is used to determine in real time whether there is a sentence segmentation mark in the latest received voice data as a real-time sentence segmentation mark recognition result; An input sentence acquisition module, for acquiring the input sentence text of the latest received input language based on the latest recognized sentence segmentation mark in the latest received voice data when the real-time sentence segmentation mark recognition result indicates that the latest received voice data contains a sentence segmentation mark; The sentence text translation module is used to determine the target translation language according to the international area code of the incoming / outgoing call, and translate the input sentence text of the latest received input language based on the target translation language to obtain real-time voice call translation results.

2. The real-time voice call translation device according to claim 1, characterized in that: The sentence segmentation mark recognition module includes: The text translation submodule is used to convert the real-time received voice data into a text data form containing pause marks to obtain real-time voice text data of the input language; The sentence segmentation mark recognition submodule is used to recognize whether there is a sentence segmentation mark in the real-time speech text data of the input language as a real-time sentence segmentation mark recognition result.

3. The real-time voice call translation device according to claim 2, characterized in that: Text translation submodule, including: A text translation unit, used to convert the voice data received in real time into a text data form to obtain the initial voice text data of the input language; A pause mark recognition unit, used for recognizing all pause marks in the voice data received in real time; The mark marking unit is used to mark all pause marks in the initial voice text data of the input language in real time to obtain the real-time voice text data of the input language.

4. The real-time voice call translation device according to claim 3, characterized in that: A pause sign recognition unit, comprising: The curve fitting subunit is used to fit the short-time energy curve of the input voice signal in real time based on the short-time energy value of the voice signal of the voice data received in real time in each preset short time period; An interval recognition subunit is used to recognize in real time all short-time energy abnormal curve intervals in the short-time energy curve of the input speech signal; The endpoint time determination subunit is used to determine the pause mark start time and the pause mark end time of all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal; A pause mark determination subunit is used to determine all pause marks in the voice data received in real time based on the pause mark start time and the pause mark end time of all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal; Among them, all abnormal short time periods of each short-time energy abnormal curve interval include a left abnormal short time period and a right abnormal short time period of each short-time energy abnormal curve interval.

5. The real-time voice call translation device according to claim 4, characterized in that: The interval identification subunit includes: A first mean value calculation terminal is used to determine all energy sudden drop curve segments in the short-time energy curve of the input speech signal, and calculate the average short-time energy value of each energy sudden drop curve segment; The second mean value calculation terminal is used to calculate the average short-time energy value of all parts of the short-time energy curve of the input voice signal except all energy sudden drop curve segments in the short-time energy curve of the input voice signal as a reference average short-time energy value; The sag ratio calculation terminal is used to calculate the ratio of the average short-time energy value of each energy sag curve segment to the reference average short-time energy value as the average energy sag ratio of each energy sag curve segment; The curve segment screening end is used to treat all energy sudden drop curve segments whose average energy sudden drop ratio is less than a preset threshold value as all short-time energy abnormal curve intervals in the short-time energy curve of the input voice signal.

6. The real-time voice call translation device according to claim 4, characterized in that: The endpoint time determination subunit includes: The endpoint period determination terminal is used to regard the preset short period corresponding to the short-time energy value at the left endpoint of each short-time energy abnormality curve interval as the left abnormal short period, and at the same time, regard the preset short period corresponding to the short-time energy value at the right endpoint of the short-time energy abnormality curve interval as the right abnormal short period; The endpoint time determination end is used to identify the start time of the pause mark in the left abnormal short time period of each short-time energy abnormal curve interval based on the left neighboring voice signal segments and the right neighboring voice signal segments of all abnormal short time periods of each short-time energy abnormal curve interval, and at the same time, identify the end time of the pause mark in the right abnormal short time period of each short-time energy abnormal curve interval.

7. The real-time voice call translation device according to claim 6, characterized in that: The endpoint moment determination end includes: The time period merging sub-terminal is used to merge each abnormal short time period of the short-time energy abnormal curve interval with the time period corresponding to the corresponding left neighboring speech signal segment and the time period corresponding to the right neighboring speech signal segment to obtain a reference time period of each short-time energy abnormal curve interval; The signal segment division sub-end is used to determine a plurality of sampling points in each abnormal short time period of each short-time energy abnormal curve interval at a preset interval, and based on each sampling point, divide the voice signal of the part of the voice data corresponding to the reference time period of the short-time energy abnormal curve interval into a left reference voice signal segment and a right reference voice signal segment corresponding to the sampling point; The zero-crossing rate difference calculating sub-terminal is used to take the difference between the zero-crossing rate of the left reference speech signal segment and the zero-crossing rate of the right reference speech signal segment at each sampling point as the left-right zero-crossing rate difference at each sampling point; The time determination sub-terminal is used to take the sampling point with the largest left and right zero-crossing rate difference among all the sampling points in the left abnormal short time period of each short-time energy abnormal curve interval as the starting time of the pause mark, and at the same time, take the sampling point with the smallest left and right zero-crossing rate difference among all the sampling points in the right abnormal short time period of each short-time energy abnormal curve interval as the starting time of the pause mark.

8. The real-time voice call translation device according to claim 2, characterized in that: The sentence segmentation mark recognition submodule includes: A word segmentation unit, used to input the real-time speech text data of the input language into the word segmentation model to obtain the word segmentation result of the real-time speech text data of the input language; A duration determination unit, used to determine the duration of all pause marks in the real-time speech text data of the input language; The sentence segmentation mark recognition and judgment unit is used to judge whether there is a sentence segmentation mark in the real-time speech text data of the input language as a real-time sentence segmentation mark recognition result based on the part of speech and meaning of each word contained in the word segmentation result and the duration of all pause marks.

9. The real-time voice call translation device according to claim 8, characterized in that: The sentence segmentation mark recognition and judgment unit includes: A model acquisition subunit is used to acquire a sentence segmentation mark recognition model; The sentence segmentation mark judgment subunit is used to input the parts of speech and meanings of all words contained in the word segmentation results and the duration of all pause marks into the sentence segmentation mark recognition model, and determine whether there are sentence segmentation marks in the real-time speech text data of the input language as the real-time sentence segmentation mark recognition result.

10. A real-time voice call translation method, characterized in that: The real-time voice call translation device as claimed in any one of claims 1 to 9 comprises: S1: Real-time determination of whether there is a sentence segmentation mark in the latest received voice data as a real-time sentence segmentation mark recognition result; S2: when the real-time segmentation mark recognition result indicates that the segmentation mark exists in the latest received voice data, based on the segmentation mark newly recognized in the latest received voice data, obtaining the input sentence text of the latest received input language; S3: Determine a target translation language according to the international area code of the incoming / outgoing call, translate the input sentence text of the latest received input language based on the target translation language, and obtain a real-time voice call translation result.

Images