Call data processing method and device, equipment, medium and product

Abstract

The invention provides a call data processing method and device, equipment, a medium and a product, and the method comprises the steps: determining a data processing task type corresponding to a call theme of call voice data according to the call voice data of two call parties in a call process, the data processing task type comprises at least one of a content generation task and an information retrieval task; performing corresponding data processing on the call voice data according to the data processing task type to obtain a data processing result; and outputting a data processing result to the terminal devices of the two parties of the call. The content of the data processing result is richer through diversified task types, so that the information processing capability in the call process is improved, and richer interaction modes are provided for the call process.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a call data processing method, device, equipment, medium and product. Background Art

[0002] Telephone communication is a common means of communication in daily life. In order to improve the richness and interactivity of calls, users can view the call transcription record after the call ends, or cast the screen in real time during the call to display the transcription text, and even render animated images based on the transcription text.

[0003] However, these methods are limited to simple text or image rendering, resulting in poor information processing capabilities during calls. Summary of the invention

[0004] Based on the above technical status, the present application provides a call data processing method, device, equipment, medium and product, which can improve the information processing effect and user experience during the call.

[0005] In order to achieve the above technical objectives, this application specifically proposes the following technical solutions:

[0006] According to the first aspect of an embodiment of the present application, a call data processing method is provided, comprising: determining a data processing task type corresponding to a call subject of the call voice data based on call voice data of two call parties during a call, the data processing task type including at least one of a content generation task and an information retrieval task; performing corresponding data processing on the call voice data according to the data processing task type to obtain a data processing result; and outputting the data processing result to terminal devices of the two call parties.

[0007] In some implementations, the call voice data includes original clauses corresponding to each of the two call parties, and determining the data processing task type corresponding to the call subject of the call voice data includes: determining the transcribed text and voice endpoints of the original clauses corresponding to each of the two call parties in the call voice data, the voice endpoints including the voice starting points; sorting the transcribed texts of the original clauses corresponding to each of the two call parties in chronological order according to the voice starting points of the original clauses corresponding to each of the two call parties to obtain a call text sequence corresponding to the call voice data; determining the data processing task type corresponding to the call subject of the call voice data according to the call text sequence corresponding to the call voice data.

[0008] In some implementations, the voice endpoint also includes a voice endpoint, and the transcribed texts of the original clauses corresponding to each of the two call parties are sorted in chronological order according to the voice starting points of the original clauses corresponding to each of the call parties to obtain a call text sequence corresponding to the call voice data, including: for each of the two call parties, if the time interval between the voice endpoint of the previous clause and the voice starting point of the latter clause in two consecutive clauses corresponding to the call party is less than or equal to a first preset time, the two consecutive clauses are merged to obtain a first round of merged clauses corresponding to the call party; the transcribed texts corresponding to the original clauses corresponding to each of the call parties and the first round of merged clauses are sorted in chronological order of the voice starting points to obtain a call text sequence corresponding to the call voice data.

[0009] In some implementations, after merging the two consecutive clauses to obtain the first-round merged clause corresponding to the caller, the method further includes: for two consecutive clauses in the original clause corresponding to the caller and the first-round merged clause, in which the time interval between the speech end point of the previous clause and the speech starting point of the next clause is greater than the first preset time and less than or equal to the second preset time, determining the semantic relevance between the previous clause and the next clause; if the semantic relevance between the previous clause and the next clause is greater than or equal to the preset relevance, merging the previous clause with the next clause to obtain the second-round merged clause corresponding to the caller; sorting the transcribed texts corresponding to the original clauses, the first-round merged clauses, and the second-round merged clauses corresponding to each of the callers in chronological order of the speech starting points to obtain a call text sequence corresponding to the call voice data.

[0010] In some implementations, determining the data processing task type corresponding to the call subject of the call voice data based on the call voice data of both parties during the call includes: determining the call subject corresponding to the call voice data based on the call voice data of both parties during the call; determining the data processing task type corresponding to the call voice data based on the call subject corresponding to the call voice data and the mapping relationship between the call subject and the data processing task type.

[0011] In some implementations, the call voice data is subjected to corresponding data processing according to the data processing task type to obtain a data processing result, including: performing corresponding data processing on the call voice data according to the data processing task type to obtain an initial data processing result; determining the material template corresponding to the call topic based on the call topic corresponding to the call voice data, and the mapping relationship between the call topic and the material template; and synthesizing the initial data processing result and the material template corresponding to the call topic to obtain a data processing result corresponding to the call voice data.

[0012] In some implementations, the call voice data is processed accordingly according to the data processing task type to obtain an initial data processing result, including: if the data processing task type is a content generation task, generating corresponding target content based on the call voice data through a content generation model; if the data processing task type is an information retrieval task, information retrieval is performed based on the call voice data through a large language model to obtain an information retrieval result.

[0013] According to the second aspect of an embodiment of the present application, a call data processing device is provided, including: a determination unit, used to determine, based on call voice data of two callers during a call, a data processing task type corresponding to a call subject of the call voice data, wherein the data processing task type includes at least one of a content generation task and an information retrieval task; a processing unit, used to perform corresponding data processing on the call voice data according to the data processing task type to obtain a data processing result; and an output unit, used to output the data processing result to terminal devices of the two callers.

[0014] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor; the memory is connected to the processor and is used to store programs; the processor is used to implement the call data processing method as described in the first aspect by running the program in the memory.

[0015] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the call data processing method as described in the first aspect is implemented.

[0016] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, cause the processor to execute: the call data processing method as described in the first aspect.

[0017] The embodiment of the present application provides a method, device, equipment, medium and product for processing call data. The method determines the call subject of the call voice data and the data processing task type corresponding to the call subject according to the conversation content in the call voice data of the two parties during the call, and then performs corresponding data processing on the call voice data according to the data processing task type to obtain the data processing result, and outputs the data processing result to the terminal devices of the two parties. Among them, the data processing task type includes at least one of a content generation task and an information retrieval task. The content generation task is used to generate corresponding target content according to the call subject related to the generative task in the conversation content, and the information retrieval task is used to generate corresponding information retrieval results according to the call subject related to the information retrieval in the conversation content. This diversified task type makes the content of the data processing result richer, can improve the information processing capability during the call, and provide a richer interactive mode for the call process. Users can not only communicate through voice, but also instantly obtain real-time generated content and retrieved information, making the call more efficient and convenient, thereby improving the user's call experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0019] Figure 1 A schematic diagram of a user dialogue process provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a sorting result of a conversation sorting method in the related art;

[0021] Figure 3 A flowchart of a call data processing method provided in an embodiment of the present application;

[0022] Figure 4 A flowchart of performing data processing according to the data processing task type provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of data processing results corresponding to the content generation task type provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of data processing results corresponding to the information retrieval task type provided in an embodiment of the present application;

[0025] Figure 7A flowchart of determining a data processing task type corresponding to a call topic of call voice data provided in an embodiment of the present application;

[0026] Figure 8 A flowchart of an implementation method for merging original clauses provided in an embodiment of the present application;

[0027] Fig. 9 A flowchart of another implementation method of merging original clauses provided in an embodiment of the present application;

[0028] Fig.10 A schematic diagram of the structure of a call data processing device provided in an embodiment of the present application;

[0029] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solution proposed in the embodiment of the present application is applicable to any communication scenario with voice calls, such as telephone communication, voice communication, video communication, etc. The technical solution of the embodiment of the present application can improve the fun and interactivity of voice calls in these application scenarios.

[0031] The technical solution provided in the embodiments of the present application can be exemplarily applied to hardware devices such as processors, electronic devices, servers (including cloud servers), or packaged into software programs to be run. When the hardware device executes the processing of the technical solution of the embodiments of the present application, or the above-mentioned software program is run, the automatic splitting of the target task and the automatic calling of the application program interface required for the task can be achieved to complete the purpose of the target task. The embodiments of the present application only exemplarily introduce the specific processing of the technical solution of the present application, and do not limit the specific implementation form of the technical solution of the present application. Any technical implementation form that can execute the processing of the technical solution of the present application can be adopted by the embodiments of the present application.

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] Before introducing this application solution, the relevant technologies are first introduced:

[0034] With the development of artificial intelligence technology, users can now experience real-time voice transcription, multi-language translation, innovative video call interface and digital human synthesis through various mobile applications (APPs). However, the realization of these functions usually requires downloading multiple APPs, which not only increases the difficulty of user selection, but also causes a large amount of mobile phone storage space to be occupied.

[0035] In recent years, with the continuous advancement of 5G network infrastructure construction, the new call infrastructure has been basically built. This progress has brought users a new way to enjoy similar functions without installing APP, greatly improving the user experience.

[0036] Based on the new call infrastructure, a variety of enhanced call functions can be implemented, such as viewing the transcription record after the call, real-time screen projection of the transcribed text during the call, and dynamic image rendering based on keywords in the transcribed text during the call.

[0037] Taking the example of dynamic image rendering based on keywords in the transcribed text during a call, the specific steps include: real-time transcription of the two voice data streams of the two parties in the voice call; matching with the preset vocabulary in the system through keyword detection technology; obtaining the corresponding image media address, and submitting the image media information to the media surface for rendering, thereby presenting an animation effect.

[0038] However, existing methods for real-time terminal rendering during calls are mostly limited to simple text or image rendering, and require media resources to be pre-set in the operator's media platform in advance. These methods lack the ability to interact dynamically, have limited media resource types, and rely solely on keyword detection and matching technology, making it difficult to meet the interactive needs of general scenarios and achieve real-time information generation, retrieval, and rendering.

[0039] In addition, according to the enterprise standard specifications for 5G new calls, the voice streams of the two parties in the new call business scenario remain independent, and the transcribed texts of the two voice streams have no direct connection at the engine level. They need to be uniformly sorted and processed at the system level based on the session identifier (ID).

[0040] Traditional conversation sorting methods usually sort by the time when the transcription results are returned by the engine. However, this method has defects when facing complex conversation situations. For example, when one party interrupts the other party, the party who finishes speaking first may be mistakenly ranked first, which reduces the accuracy of call transcription and affects the quality of the call content finally presented on the user terminal, thereby reducing the user's call experience. The following is an illustration of the traditional conversation sorting method with reference to the accompanying figure:

[0041] Figure 1A schematic diagram of a user dialogue process provided in an embodiment of the present application. Figure 1 As shown in the figure, assume that user A and user B are talking on the phone. A-1-1, B-1-1, A-2-1, B-2-1, A-3-1, B-3-1, A-4-1, B-4-1, and A-4-2 are conversations between user A and user B. The following scenarios may exist during the conversation:

[0042] Scenario 1: User A speaks first and finishes first.

[0043] Scenario 2: User-A speaks first and then finishes.

[0044] Scenario 3: User A speaks first and finishes first, and user B interrupts while user A is speaking.

[0045] Scenario 4: User-B interrupts between two consecutive sentences of User-A.

[0046] For the above dialogue process, the ranking results of the traditional dialogue ranking method are as follows: Figure 2 As shown, it can be seen that for the above scenario 2), since user-B interrupted user-A while he was speaking, but user-B finished speaking first, the system would receive user-B's call transcription first, resulting in sentence B-2-1 being ranked first. For scenario 4), since user-B interrupted user-A while he was speaking continuously, such as "OK, OK", but because the system received user-B's speech first, user-B's sentence B-4-1 would cut off A's continuous speech (A-4-1 and A-4-2).

[0047] In summary, the traditional conversation sorting method may cause the speaker who spoke last but finished speaking first to be ranked first, resulting in low accuracy of call transcription, which in turn affects the final presentation of the call content on the user's terminal device and reduces the user's call experience.

[0048] In view of this, the embodiments of the present application are dedicated to providing a call data processing method, device, equipment, medium and product, which can not only improve the fun and interactivity of calls, but also make the user's communication experience richer and more diverse. At the same time, users do not need to worry about the number of application software and storage space issues, thereby providing users with a richer and more diverse communication experience. The following embodiments are described in detail one by one.

[0049] Exemplary Methods

[0050] Figure 3 A flowchart of a call data processing method provided in an embodiment of the present application. Figure 3As shown, the call data processing method provided in this embodiment includes steps S101 to S103:

[0051] S101. Determine a data processing task type corresponding to a call topic of the call voice data according to call voice data of both parties during a call.

[0052] The call voice data refers to the real-time voice stream data generated when the two parties (including the calling party and the called party) are talking, which includes two independent real-time voice stream data of the calling party and the called party.

[0053] The call voice data contains the call topics corresponding to the chat content of the two parties during the call. The call topics can be the main topics or categories of the chat content, such as weather, attractions, food, navigation, poetry, etc. By determining the call topics in the call voice data and generating richer content based on these call topics, and then presenting them on the terminal devices of the two parties, the fun and interactivity of the user's call can be improved.

[0054] The specific implementation of step S101 includes the following steps a1 and a2:

[0055] Step a1: based on the call voice data of both parties during the call, determine the call topic corresponding to the call voice data.

[0056] When the call voice data of both parties is received, the call voice data is transcribed, and a call transcription text can be generated to facilitate the subsequent determination of the call subject.

[0057] In order to improve the efficiency of call data processing and present real-time output on the user's terminal device, the voice transcription operation can be performed in real time during the user's call.

[0058] After obtaining the call transcription text, semantic analysis can be performed based on the call transcription text to determine the call subject of the call voice data.

[0059] In some embodiments, the call topic can be determined by a call topic prediction model. The call transcription text is input into the call topic prediction model, which is combined with the contextual dialogue text in the call transcription text to determine the call topic to which the user chat content belongs. For example, if the call transcription text mentions "Today's weather is really good", the model can determine that the call topic is "weather".

[0060] The call topic prediction model can be obtained by training a large language model (such as a 2.6B large language model) based on call transcription text samples and corresponding call topics. The call transcription text samples can include samples of various common call topics, such as weather, attractions, food, navigation, poetry, etc.

[0061] Step a2: Determine the data processing task type corresponding to the call voice data according to the call subject corresponding to the call voice data and the mapping relationship between the call subject and the data processing task type.

[0062] Specifically, a mapping relationship between a call topic and a data processing task type can be pre-established. For example, a mapping relationship between weather and information retrieval tasks, scenic spots and information retrieval tasks, food and content generation tasks, navigation and information retrieval tasks, poetry and content generation tasks. The mapping relationship can be stored in a configuration file or database for quick query and matching.

[0063] Afterwards, according to the determined call subject, the pre-established mapping relationship is queried to determine the data processing task type corresponding to the call voice data, wherein the data processing task type includes at least one of a content generation task and an information retrieval task.

[0064] For example, if the topic of the call is "poetry", the data processing task type is determined to be a content generation task according to the mapping relationship. If the topic of the call is "weather", the data processing task type is determined to be an information retrieval task according to the mapping relationship.

[0065] S102: Perform corresponding data processing on the call voice data according to the data processing task type to obtain a data processing result.

[0066] Figure 4 A flowchart of performing data processing according to the data processing task type provided in the embodiment of the present application. Figure 4 As shown, the specific implementation of step S102 includes the following steps S401, S402 and S403:

[0067] S401. Perform corresponding data processing on the call voice data according to the data processing task type to obtain an initial data processing result.

[0068] The initial data processing result includes the target content corresponding to the content generation task and the information retrieval result corresponding to the information retrieval task. The specific implementation of step S401 includes the following steps b1 and b2:

[0069] Step b1: If the data processing task type is a content generation task, a content generation model is used to generate corresponding target content based on the call voice data.

[0070] In this step, the content generation model is an artificial intelligence generated content (AIGC) model, which can automatically generate various contents such as text, images, audio, video, etc. according to the call topic.

[0071] For example, when a parent calls a child to find out what they are currently learning at school, the child mentions that the teacher taught the ancient poem "Viewing the Waterfall at Mount Lu". The call topic prediction model predicts that the call topic is "poetry", and then, based on the mapping relationship between the call topic and the data processing task type, determines that the data processing task type corresponding to "poetry" is a content generation task. Next, the prompt words for content generation are automatically generated, and the prompt words for content generation are input into the content generation model so that it generates poetry content based on the call topic in the call voice data. The prompt words for content generation can be specifically as follows:

[0072] You are a poetry expert, and your main task is to find information related to poetry from the user's conversation context and complete it.

[0073] The completed information includes: <poem name>, <author>, <dynasty>, <category>, and <content>.

[0074] Output format:

[0075] 1. Output the result in json format: {"code":"0000","data":{"name":{name},"author":{author},"dynasty":{dynasty},"category":{category},"content":{content}}}

[0076] 2. Use the symbol "#" to separate sentences in the content of the poem

[0077] 3. If none of the above four key information is extracted, the output content is:

[0078] {"code":"1000"}.

[0079] The following is a reference example. The content in double quotes is the user's conversation content:

[0080] "<Other> Mom, before school is over this afternoon, the teacher has arranged something for us: we need to memorize an ancient poem, and the teacher will conduct a random check on us during class on Monday.

[0081] Have you decided which poem you want to recite?

[0082] <The other party>I want to recite "Ascending the Stork Tower".

[0083] The result you need to output is: {"code": "0000", "data": {"name": "Ascending the Stork Tower", "author": "Wang Zhihuan", "dynasty": "Tang Dynasty", "content": "The sun along the mountain bows; The Yellow River seawards flows. #You can enjoy a great sight; By climbing to a greater height."}}

[0084] Please refer to the above example to output the result for the following conversation content: "<The other party>Mom, before school was over this afternoon, the teacher arranged one thing: to memorize a classical Chinese poem thoroughly. The teacher will conduct spot checks during class on Monday.\n<Me>Have you decided which poem to recite?\n<The other party>I want to recite \"Viewing the Lu Mountain Waterfall\"."

[0085] Step b2: If the data processing task type is an information retrieval task, then use the large language model to perform information retrieval based on the call voice data to obtain the information retrieval result.

[0086] For example, when friends are discussing where to go during holidays in a call and mention a certain scenic spot (such as: Huangshan Mountain), then the call topic prediction model predicts the call topic as "tourist attraction". Then, according to the mapping relationship between the call topic and the data processing task type, it is determined that the data processing task type corresponding to "tourist attraction" is an information retrieval task. Next, an information retrieval prompt word is automatically generated, and the information retrieval prompt word is input into the content generation model, enabling it to extract the specific scenic spot name and the information of the city it belongs to from the call topic in the call voice data, and then call relevant information source services such as scenic spots and weather to obtain the situation of the scenic spot and the recent weather conditions. The information retrieval prompt word can be specifically as follows:

[0087] You are an algorithm expert, good at analyzing and extracting location information related to the weather from a conversation, such as: city, region, etc.

[0088] When extracting information, the following two conditions need to be met:

[0089] 1. It is necessary to remove the appellation words related to people.

[0090] 2. The extracted information includes <the city name city and region name area concerned in the conversation>

[0091] You need to output the result according to the following requirements:

[0092] 1. Extract the key information in this conversation and output the result in json format: {"code": "0000", "data": {"city": [city], "area": [area]}}

[0093] 2. If the city name is not extracted, the output content is: {"code":"1000"}

[0094] The following is a reference example. The content in double quotes is the user's conversation content:

[0095] “<Other Party> The weather in Wuhan is too hot today.

[0096] Yes, the weather forecast says it will get colder tomorrow.

[0097] <Other> Really? That’s great! ”

[0098] The result you need to output is: {"code":"0000","data":{"city":"Hefei","area":""}}

[0099] Please refer to the above example to output the result of the following conversation: "<other party> The weather in Hefei is too hot today\n<me> Yes, the weather forecast says it will get cooler tomorrow\n<other party> Really? That would be great".

[0100] Continue reading Figure 4 , after step S401, step S402 may also be included.

[0101] S402: Determine a material template corresponding to the call subject based on the call subject corresponding to the call voice data and a mapping relationship between the call subject and the material template.

[0102] After the call subject is determined, a material template suitable for the current call subject can be selected according to the pre-established mapping relationship between the call subject and the material template.

[0103] A material template is a document or design framework with a fixed format and standardized structure, which is mainly used to simplify the creation of regular documents. It provides a common document template so that documents with similar structure and appearance can be quickly generated. Material templates not only include basic layout designs, but also include professional terms, chart styles, color matching and other elements in specific fields, which can be customized according to different call themes. For example, if the call theme is a business meeting, you may choose a business template that contains professional terms, charts and company logos; if it is an educational consultation, you may adopt an education-oriented design style.

[0104] In this way, the appropriate material template can be automatically matched for each call, which not only improves data processing efficiency, but also ensures the consistency in form of the generated documents or content, and makes the content presented on the user's terminal device richer.

[0105] S403: synthesize the initial data processing result and the material template corresponding to the call subject to obtain the data processing result corresponding to the call voice data.

[0106] After determining the material template corresponding to the call topic, the initial data processing results and the selected material template can be synthesized. The synthesis process not only includes filling the text information into the template, but also involves more complex integration operations to ensure that the final generated content can reflect the original call data and meet the design requirements of the material template.

[0107] Specifically, the key points extracted from the call can be filled in according to the structure and format of the material template. For example, for a business meeting call, the meeting summary, discussion points, follow-up action plan and other information can be filled in the corresponding preset positions in the template. At the same time, some variable elements in the template, such as chart data, background pictures, etc., can be automatically adjusted according to the nature of the call content to better reflect the actual content of the call.

[0108] Figure 5 Schematic diagram of data processing results corresponding to the content generation task type provided in the embodiment of the present application. Figure 5 As shown, we can see that the poetry content is the generated target content. Figure 5 The background image in the material template is the material template. By inserting the poem content into the corresponding position in the material template, the final data processing result can be obtained.

[0109] Figure 6 Schematic diagram of data processing results corresponding to the information retrieval task type provided in the embodiment of the present application. Figure 6 As shown, the specific weather (such as "cloudy" or "sunny"), specific temperature (such as "10℃~20℃", "9℃~21℃", "10℃~22℃") and air quality (such as good air and slightly polluted air) of the city "Hefei" are the initial data processing results. The layout design in the figure and the symbols of cloudy and sunny days are the material templates. By inserting the specific weather (such as "cloudy" or "sunny"), specific temperature (such as "10℃~20℃", "9℃~21℃", "10℃~22℃") and air quality (such as good air and slightly polluted air) of "Hefei" into the corresponding positions in the material template, the final data processing results can be obtained.

[0110] according to Figure 5 and Figure 6It can be seen that during a call, users can intuitively see real-time information such as their daily life and knowledge encyclopedias, avoiding the need to open the APP and perform various tedious operations to achieve the above functions, thereby improving call efficiency and assisting users in making communication decisions, providing convenience for users.

[0111] Continue reading Figure 3 , after step S102, step S103 may also be included.

[0112] S103: Output the data processing result to the terminal devices of both parties of the call.

[0113] Specifically, the initial data processing results and material templates are synthesized into an RTP video stream, which is pushed to the operator's media interface platform through the operator's media capability platform and finally rendered on the user terminal, thereby providing users with richer interaction methods during the call.

[0114] As described in the above embodiment, when determining the data processing task type corresponding to the call subject of the call voice data, it is necessary to first transcribe the call voice data into a call transcription text, and then determine the data processing task type corresponding to the call subject based on the call transcription text. In order to improve the accuracy of the call transcription text dialogue sorting, this embodiment adopts the following specific implementation method:

[0115] Figure 7 A flowchart of determining the data processing task type corresponding to the call subject of the call voice data provided in the embodiment of the present application. Figure 7 As shown, the call voice data includes original clauses corresponding to the two parties of the call, and determining the data processing task type corresponding to the call subject of the call voice data includes the following steps S701 to S703:

[0116] S701. Determine the transcribed text and voice endpoints of the original clauses corresponding to the two parties in the call voice data, where the voice endpoints include the voice starting point.

[0117] Specifically, it is necessary to perform endpoint detection on the caller and the called party's respective call voice data through a voice endpoint detection method to determine each voice starting point and each voice end point in the call voice data. The voice starting point and voice end point refer to the start time and end time of each original clause contained in the call voice data.

[0118] Voice endpoint detection methods include methods based on short-time energy and zero-crossing rate, threshold-based voice activity detection (VAD), classifier-based voice endpoint detection methods, and model-based voice endpoint detection methods.

[0119] Through these voice endpoint detection methods, the voice starting point and voice end point of each original clause in the call voice data can be accurately identified. Furthermore, the call voice data can be divided into clauses according to the identified voice starting points and voice end points. In addition, the original clauses of the caller and the called party in the call voice data are transcribed respectively to obtain the call transcribed texts of the caller and the called party.

[0120] S702. Sort the transcribed texts of the original clauses corresponding to the two parties of the call in chronological order according to the speech starting points of the original clauses corresponding to the two parties of the call, to obtain a call text sequence corresponding to the call voice data.

[0121] After determining the call transcribed text and voice endpoints corresponding to each party of the call, the call transcribed texts of the caller and the called party can be uniformly sorted in chronological order according to the voice starting point of each original clause in the call voice data to generate a call text sequence corresponding to the call voice data, thereby ensuring that the final generated text can accurately reflect the actual speaking order of the two parties of the call.

[0122] The speech start point is the timestamp of the beginning of each clause and is the key basis for sorting. The speech start point time of all original clauses and their corresponding call transcription texts are combined into a list. Each list item contains two parts: the speech start point time and the corresponding call transcription text. For example, a list item can be expressed as [speech start point time, call transcription text]. After that, the list is sorted in ascending order according to the speech start point time through a sorting algorithm to ensure that the transcription text is arranged in the order of the actual call. Sorting algorithms can include quick sort, merge sort, etc.

[0123] After the sorting is completed, the transcription texts in the list are connected in order to obtain the call text sequence. Specifically, each transcription text is arranged in the order in which it actually appears in the call, which can ensure the coherence and logic of the text content.

[0124] Through the above steps, the call transcription texts of both parties can be sorted in chronological order to generate a complete and orderly call text sequence.

[0125] In some scenarios, users may pause briefly while speaking, causing the voice endpoint detection method to recognize a complete sentence as two clauses, which is inconsistent with the actual call situation. To avoid this problem, the original clauses of the caller and the called party can be merged separately, and sorted based on the merged clauses to ensure that the final result is more consistent with the actual call situation. The specific implementation method is as follows:

[0126] Figure 8A flowchart of an implementation method for merging original clauses provided in an embodiment of the present application.

[0127] like Figure 8 As shown, step S702 includes step S801 and step S802:

[0128] S801. For each party in the call, if, in two consecutive original clauses corresponding to the party, the time interval between the speech end point of the previous original clause and the speech starting point of the next original clause is less than or equal to a first preset time, the two consecutive original clauses are merged to obtain a first round of merged clauses corresponding to the party.

[0129] In step S801, it is necessary to traverse the consecutive original clauses of each of the two parties in the conversation to determine whether they need to be merged.

[0130] Specifically, for each caller, the time interval between the speech end point of the previous original clause and the speech start point of the next original clause is traversed between each two consecutive clauses in all original clauses. If the time interval between two consecutive clauses is less than or equal to the first preset time, the two consecutive original clauses are merged to obtain a first round of merged clauses. The speech start point of the first round of merged clauses is updated to the speech start point of the previous original clause, and the speech end point is updated to the speech end point of the next original clause.

[0131] Similarly, the original clause of the called party and the first round of merged clauses can be obtained.

[0132] S802. Sort the original clauses corresponding to the two parties of the call and the transcribed texts corresponding to the first-round merged clauses according to the time sequence of the speech starting points to obtain a call text sequence corresponding to the call voice data.

[0133] After completing the first round of clause merging, the original clauses and merged clauses of both parties of the call need to be sorted together to generate the final call text sequence.

[0134] The following describes step S801 and step S802 in detail by taking an example:

[0135] For example, assuming that the first preset time is 250 milliseconds, the original clause of the calling party A includes the following three original clauses, in the format of [speech start point, speech end point, "specific content of the original clause"]:

[0136] Clause A-1: ​​[1000ms, 1200ms, "Hello"]

[0137] Clause A-2: [1250 ms, 1500 ms, "I am Zhang San"]

[0138] Clause A-3: [2000 ms, 2500 ms, "The weather in Hefei is nice today"]

[0139] The original clause of the called party B includes the following three original clauses:

[0140] Clause B-1: [1500 ms, 1800 ms, "Hello, Zhang San"]

[0141] Clause B-2: [1900ms, 2200ms, "I am John Doe"]

[0142] Clause B-3: [2600ms, 3000ms, "Yes, the weather is nice"]

[0143] After the first round of merging clauses, the caller's clause is updated to:

[0144] Merge clauses A-1 & A-2: [1000 ms, 1500 ms, "Hello, I am Zhang San"]

[0145] Clause A-3: [2000 ms, 2500 ms, "The weather in Hefei is nice today"]

[0146] After the first round of merging clauses, the clause of the called party is updated to:

[0147] Merge clauses B-1 & B-2: [1500 ms, 2200 ms, "Hello, Zhang San, I am Li Si"]

[0148] Clause B-3: [2600ms, 3000ms, "Yes, the weather is nice"]

[0149] The sorted call text sequence is:

[0150] Merge clauses A-1 & A-2: Hello, I am Zhang San.

[0151] Clauses B-1&B-2: Hello, Zhang San, I am Li Si.

[0152] Clause A-3: The weather in Hefei is nice today.

[0153] Clause B-3: Yes, the weather is nice.

[0154] Through the above steps, the final generated call text sequence not only retains the original content of each clause, but also arranges it in chronological order according to the actual call, ensuring the coherence and accuracy of the text.

[0155] In some embodiments, in order to further improve the accuracy of conversation sorting of call transcription text, such as Fig. 9 As shown, after step S802, this embodiment may further include steps S803 to S805:

[0156] S803. For two consecutive clauses in the original clause and the first-round merged clauses corresponding to the caller, in which the time interval between the speech end point of the previous clause and the speech starting point of the next clause is greater than the first preset time and less than or equal to the second preset time, determine the semantic relevance between the previous clause and the next clause.

[0157] The second preset time may be 1000 milliseconds.

[0158] In this embodiment, the clauses of the call party need to be further analyzed, such as two consecutive clauses whose time interval is greater than the first preset time and less than or equal to the second preset time. For each call party, all two consecutive clauses are traversed, and if the time interval between the speech end point of the previous clause and the speech start point of the next clause is greater than or equal to 250 milliseconds and less than or equal to 1000 milliseconds, the pair of clauses is marked as a clause pair that needs further analysis.

[0159] Then, natural language processing techniques (such as semantic similarity calculation, word vector model, etc.) are used to determine the semantic relevance between the previous clause and the next clause. For example, cosine similarity can be used to calculate the semantic similarity of two clauses.

[0160] S804: If the semantic relevance between the previous clause and the next clause is greater than or equal to the preset relevance, the previous clause and the next clause are merged to obtain a second round of merged clauses corresponding to the caller.

[0161] If the semantic relevance between the previous clause and the next clause is greater than or equal to a preset relevance (eg, 0.7), the two clauses are merged. The merged clause contains all the contents of the two clauses and updates its speech start point and speech end point.

[0162] S805. Sort the original clauses, the first-round merged clauses, and the transcribed texts corresponding to the two parties of the call in chronological order of the speech starting points to obtain a call text sequence corresponding to the call voice data.

[0163] Specifically, the original clauses of both parties of the call, the first-round merged clauses, and the second-round merged clauses are collected together to form a list containing all clauses. Each list item contains the speech start time of the clause and the corresponding transcribed text. The list is sorted in ascending order according to the speech start time to ensure that the transcribed text is arranged in the order of the actual call. After the sorting is completed, the transcribed texts in the list are connected in order to form the final call text sequence.

[0164] In some embodiments, after the sorting is completed in step S802 or step S805, the call text sequence of each call party may be traversed, and two consecutive clauses in which the other party has not spoken may be merged to obtain a final call text sequence.

[0165] In some embodiments, in order to optimize the interactive experience, for content generation scenarios, real-time synthesis can be performed when the large model responds for the first time (such as when the large model outputs the first word in the poem content) to reduce the interaction delay.

[0166] In addition, during the formal streaming process, the video bit rate and the frequency of occurrence of I frames (Group of Pictures, GOP) in video encoding can be dynamically adjusted in real time based on the Real-time Control Protocol (RTCP) information returned by the operator's media interface to ensure smooth rendering on the user terminal.

[0167] Continue reading Figure 7 , after step S702, step S703 may also be included.

[0168] S703: Determine the data processing task type corresponding to the call subject of the call voice data according to the call text sequence corresponding to the call voice data.

[0169] As described above, the call topic can be determined by the call topic prediction model. The call transcription text is input into the call topic prediction model, which is combined with the contextual dialogue text in the call transcription text to determine the call topic to which the user chat content belongs.

[0170] In summary, this embodiment has the following beneficial effects:

[0171] (1) By analyzing the conversation content in the voice data of the two parties during the call, the topic of the call can be accurately identified, and the type of data processing task corresponding to the topic of the call can be determined to ensure the pertinence and effectiveness of subsequent data processing.

[0172] (2) The data processing task type includes at least one of a content generation task and an information retrieval task. The content generation task can generate corresponding target content based on the call topic related to the generative task in the conversation content. The information retrieval task can generate corresponding information retrieval results based on the call topic related to information retrieval in the conversation content, such as querying the weather, searching for information, etc. Through content generation tasks and information retrieval tasks, a richer interactive mode is provided for the call process. This diversified task type makes the data processing results richer and improves the information processing capabilities and user experience during the call process. Users can not only communicate through voice, but also obtain real-time generated content and retrieved information, making calls more efficient and convenient.

[0173] (3) By processing the call voice data accordingly according to the determined data processing task type, accurate data processing results are generated, and the data processing results are output to the terminal devices of both parties in real time. This ensures the timeliness and accuracy of the output data processing results, enhancing the interactivity and user experience of the call.

[0174] (4) Through automated and intelligent data processing, the need for manual operations by users is reduced, simplifying the process of information acquisition and content generation. Users can instantly obtain the required information or generate content during a call, thereby improving the user's call experience. At the same time, users do not need to worry about the number of application software and storage space issues, thus providing users with a richer and more diverse communication experience.

[0175] (5) This method can be applied to a variety of call scenarios, such as business meetings, education and training, customer service, etc., and can meet the needs of different scenarios. For example, in business meetings, meeting minutes can be automatically generated; in education and training, explanations and supplementary materials of relevant knowledge points can be provided in real time; in customer service, solutions can be quickly retrieved and provided.

[0176] Exemplary Devices

[0177] Corresponding to the above-mentioned call data processing method, the embodiment of the present application also provides a call data processing device. Fig.10 Schematic diagram of a call data processing device provided by an embodiment of the present application. Fig.10As shown, the call data processing device provided in the embodiment of the present application includes: a determination unit 1001, a processing unit 1002 and an output unit 1003; wherein the determination unit 1001 is used to determine the data processing task type corresponding to the call subject of the call voice data according to the call voice data of the call parties during the call, and the data processing task type includes at least one of a content generation task and an information retrieval task; the processing unit 1002 is used to perform corresponding data processing on the call voice data according to the data processing task type to obtain a data processing result; the output unit 1003 is used to output the data processing result to the terminal devices of the call parties.

[0178] In some embodiments, the call voice data includes original clauses corresponding to each of the two call parties, and the determination unit 1001 determines the data processing task type corresponding to the call subject of the call voice data, including: determining the transcribed text and voice endpoint of the original clauses corresponding to each of the two call parties in the call voice data, and the voice endpoint includes a voice starting point; according to the voice starting points of the original clauses corresponding to each of the two call parties, sorting the transcribed texts of the original clauses corresponding to each of the two call parties in chronological order to obtain a call text sequence corresponding to the call voice data; according to the call text sequence corresponding to the call voice data, determining the data processing task type corresponding to the call subject of the call voice data.

[0179] In some embodiments, the voice endpoint also includes a voice endpoint, and the determination unit 1001 sorts the transcribed texts of the original clauses corresponding to each of the two call parties in chronological order according to the voice starting points of the original clauses corresponding to each of the call parties, to obtain a call text sequence corresponding to the call voice data, including: for each of the two call parties, if the time interval between the voice endpoint of the previous clause and the voice starting point of the latter clause in two consecutive clauses corresponding to the call party is less than or equal to a first preset time, then the two consecutive clauses are merged to obtain a first round of merged clauses corresponding to the call party; the transcribed texts corresponding to the original clauses corresponding to each of the call parties and the first round of merged clauses are sorted in chronological order of the voice starting points to obtain a call text sequence corresponding to the call voice data.

[0180] In some embodiments, the determination unit 1001 is also used to: for two consecutive clauses in the original clause corresponding to the caller and the first-round merged clause, in which the time interval between the speech end point of the previous clause and the speech starting point of the next clause is greater than the first preset time and less than or equal to the second preset time, determine the semantic relevance between the previous clause and the next clause; if the semantic relevance between the previous clause and the next clause is greater than or equal to the preset relevance, merge the previous clause with the next clause to obtain the second-round merged clause corresponding to the caller; sort the transcribed texts corresponding to the original clauses, the first-round merged clauses and the second-round merged clauses corresponding to each of the callers in chronological order of the speech starting points to obtain a call text sequence corresponding to the call voice data.

[0181] In some embodiments, the determination unit 1001 determines the data processing task type corresponding to the call subject of the call voice data based on the call voice data of the call parties during the call, including: determining the call subject corresponding to the call voice data based on the call voice data of the call parties during the call; determining the data processing task type corresponding to the call voice data based on the call subject corresponding to the call voice data, and the mapping relationship between the call subject and the data processing task type.

[0182] In some embodiments, the processing unit 1002 performs corresponding data processing on the call voice data according to the data processing task type to obtain a data processing result, including: performing corresponding data processing on the call voice data according to the data processing task type to obtain an initial data processing result; determining the material template corresponding to the call topic based on the call topic corresponding to the call voice data, and the mapping relationship between the call topic and the material template; synthesizing the initial data processing result and the material template corresponding to the call topic to obtain the data processing result corresponding to the call voice data.

[0183] In some embodiments, the processing unit 1002 performs corresponding data processing on the call voice data according to the data processing task type to obtain an initial data processing result, including: if the data processing task type is a content generation task, then generating corresponding target content based on the call voice data through a content generation model; if the data processing task type is an information retrieval task, then performing information retrieval based on the call voice data through a large language model to obtain an information retrieval result.

[0184] The call data processing device provided in this embodiment belongs to the same application concept as the call data processing method provided in the above embodiments of this application, can execute the call data processing method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the call data processing method. For technical details not fully described in this embodiment, please refer to the specific processing content of the call data processing method provided in the above embodiments of this application, which will not be repeated here.

[0185] The functions implemented by the above-mentioned determination unit 1001, processing unit 1002 and output unit 1003 may be implemented by the same or different processors respectively, which is not limited in the embodiment of the present application.

[0186] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0187] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0188] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0189] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0190] Exemplary Electronic Devices

[0191] The present application embodiment provides an electronic device, see Fig.11 As shown, the electronic device includes:

[0192] Memory 200 and processor 210;

[0193] The memory 200 is connected to the processor 210 and is used to store programs;

[0194] The processor 210 is used to implement the call data processing method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0195] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0196] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0197] A bus may include a pathway that transfers information between components of a computer system.

[0198] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0199] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0200] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0201] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0202] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0203] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0204] The processor 210 executes the program stored in the memory 200 and calls other devices, and can be used to implement each step of any call data processing method provided in the above embodiments of the present application.

[0205] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the call data processing method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the introduction to the embodiment of the above-mentioned call data processing method.

[0206] Exemplary computer program products and storage media

[0207] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the call data processing method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0208] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0209] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the call data processing method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0210] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0211] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0212] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0213] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0214] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0215] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0216] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0217] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0218] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0219] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0220] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A call data processing method, characterized in that: include: Determine, according to the call voice data of the two parties during the call, a data processing task type corresponding to the call subject of the call voice data, wherein the data processing task type includes at least one of a content generation task and an information retrieval task; According to the data processing task type, the call voice data is processed accordingly to obtain a data processing result; The data processing result is output to the terminal devices of the two parties of the call.

2. The method according to claim 1, characterized in that The call voice data includes original clauses corresponding to the two parties of the call, and the determining of the data processing task type corresponding to the call subject of the call voice data includes: Determine the transcribed text and voice endpoints of the original clauses corresponding to the two parties of the call in the call voice data, wherein the voice endpoints include the voice starting points; According to the speech starting points of the original clauses corresponding to the two parties of the call, the transcribed texts of the original clauses corresponding to the two parties of the call are sorted in chronological order to obtain a call text sequence corresponding to the call voice data; According to the call text sequence corresponding to the call voice data, a data processing task type corresponding to the call subject of the call voice data is determined.

3. The method according to claim 2, characterized in that The voice endpoint also includes a voice endpoint, and the transcribed texts of the original clauses corresponding to the two parties of the call are sorted in chronological order according to the voice starting points of the original clauses corresponding to the two parties of the call, to obtain a call text sequence corresponding to the call voice data, including: For each of the two parties of the conversation, if, in two consecutive clauses corresponding to the party, the time interval between the speech end point of the previous clause and the speech start point of the next clause is less than or equal to a first preset time, the two consecutive clauses are merged to obtain a first round of merged clauses corresponding to the party of the conversation; The original clauses corresponding to the two parties of the call and the transcribed texts corresponding to the first-round merged clauses are sorted in chronological order of the starting points of the speech to obtain a call text sequence corresponding to the call voice data.

4. The method according to claim 3, characterized in that After merging the two consecutive clauses to obtain a first round of merged clauses corresponding to the calling party, the method further includes: For two consecutive clauses in the original clause corresponding to the caller and the first-round merged clauses, where the time interval between the speech end point of the previous clause and the speech start point of the next clause is greater than the first preset time and less than or equal to the second preset time, determine the semantic relevance between the previous clause and the next clause; If the semantic relevance between the previous clause and the next clause is greater than or equal to a preset relevance, the previous clause and the next clause are merged to obtain a second round of merged clauses corresponding to the caller; The original clauses corresponding to the two parties of the call, the transcribed texts corresponding to the first-round merged clauses and the second-round merged clauses are sorted in chronological order of the starting points of the speech to obtain a call text sequence corresponding to the call voice data.

5. The method according to any one of claims 1 to 4, characterized in that The determining, according to the call voice data of the two parties during the call, the data processing task type corresponding to the call subject of the call voice data includes: Determine, based on the call voice data of both parties during the call, a call topic corresponding to the call voice data; The data processing task type corresponding to the call voice data is determined according to the call subject corresponding to the call voice data and the mapping relationship between the call subject and the data processing task type.

6. The method according to claim 5, characterized in that The performing corresponding data processing on the call voice data according to the data processing task type to obtain a data processing result includes: Performing corresponding data processing on the call voice data according to the data processing task type to obtain an initial data processing result; Determine the material template corresponding to the call theme based on the call theme corresponding to the call voice data and the mapping relationship between the call theme and the material template; The initial data processing result and the material template corresponding to the call subject are synthesized to obtain a data processing result corresponding to the call voice data.

7. The method according to claim 6, characterized in that The performing corresponding data processing on the call voice data according to the data processing task type to obtain an initial data processing result includes: If the data processing task type is a content generation task, generating target content corresponding to the call voice data through a content generation model; If the data processing task type is an information retrieval task, information retrieval is performed based on the call voice data using a large language model to obtain an information retrieval result.

8. A call data processing device, characterized in that: include: a determination unit, configured to determine, according to the call voice data of the two callers during the call, a data processing task type corresponding to the call subject of the call voice data, wherein the data processing task type includes at least one of a content generation task and an information retrieval task; A processing unit, configured to perform corresponding data processing on the call voice data according to the data processing task type to obtain a data processing result; The output unit is used to output the data processing result to the terminal devices of the two parties of the call.

9. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 7 by running the program in the memory.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, cause the processor to implement the method as claimed in any one of claims 1 to 7.

Images