Video calling method and system based on deep learning

Abstract

The embodiment of the present application provides a deep learning-based video call method and system, which obtains pre-communication information between the target user and the pre-communication agent, and the user information of the target user in the pre-communication stage. According to the pre-communication audio, the business demand information of the target user is obtained. According to the business demand information, a text message link for triggering the video call address of the video agent is constructed. If a call request for a video call to a video agent is received, the video agent is called to establish a video call between the terminal device of the target user and the video agent in the formal communication stage, and the formal communication interaction between the video agent and the target user is realized based on a specific video mode and a pre-filled business data document generated based on the business demand information, thereby utilizing the processing of the pre-communication information to pre-provide the key content of the pre-communication information to the video agent in the formal communication stage, thereby reducing the situation of repeated description and improving the processing efficiency of business processing.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a video calling method and system based on deep learning. Background Art

[0002] In the traditional business service model, telephone seats and video seats are separated. This separation mode brings many inconveniences in actual business operations. When the service personnel access the telephone seat to seek service, the telephone seat cannot directly complete some specific services due to the limitations of its own functions or permissions. For example, for some complex financial services or services that require visual operation guidance, the telephone seat staff can only tell the service personnel to go to a specific channel to re-initiate the business request. This process not only increases the complexity of the service personnel's operations, but also easily leads to the loss or misunderstanding of information during the transmission process. The service personnel need to re-describe the business requirements to the video seat staff, and this repeated description is a waste of time and energy, which seriously affects the efficiency of the entire business process.

[0003] Therefore, how to improve the processing efficiency of business services is an urgent problem to be solved. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a video calling method and system based on deep learning to improve the processing efficiency of business services.

[0005] In conjunction with the first aspect of the present application, a deep learning-based video calling method is provided, the method comprising: In the pre-communication stage, pre-communication information between the target user and the pre-communication agent and user information of the target user are obtained, wherein the pre-communication information includes pre-communication audio and the time of the pre-communication audio; Acquiring the business demand information of the target user according to the pre-communication audio; Obtaining the identification of the video seat in the formal communication stage, and the video mode in the formal communication stage, wherein the video mode includes digital human video communication; Constructing a text message link according to the service demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio, and sending the text message link to the terminal device of the target user according to the user information of the target user, wherein the text message link includes a video call address for triggering a video call to the video agent; If a call request for a video call to the video agent is received, calling the video agent to establish a video call between the terminal device of the target user and the video agent in the formal communication stage; Determine a target transaction corresponding to the business demand information according to the business demand information, and fill in a business data pre-filled template of the target transaction according to the business demand information to generate a pre-filled business data document; Based on the video mode and the pre-filled business data document, formal communication interaction is achieved between the video agent and the target user.

[0006] In a possible implementation of the first aspect, constructing the SMS link according to the business demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio includes: Generate a primary key identifier of the SMS link according to the time of the pre-communication audio; Generate the content of the SMS link and the video call address according to the identifier of the video agent, the video mode, and the service demand information; The SMS link is constructed according to the primary key identifier of the SMS link, the content of the SMS link, and the video call address.

[0007] In a possible implementation of the first aspect, the present invention further includes: Get the preset call time limit; The step of constructing a text message link according to the business demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio includes: Generate a primary key identifier of the SMS link according to the time of the pre-communication audio; Generate the content of the SMS link and the video call address according to the identifier of the video agent, the video mode, and the service demand information; Constructing the SMS link according to the primary key identifier of the SMS link, the content of the SMS link, the video call address, and the preset call time limit; If a call request for a video call to the video agent is received, calling the video agent comprises: If a call request for a video call to the video agent is received within the preset call time limit, calling the video agent; If a call request for a video call to the video agent is received outside the preset call time limit, the video agent is not called.

[0008] In a possible implementation manner of the first aspect, acquiring the service demand information of the target user according to the pre-communication audio includes: Performing frame processing on the pre-communication audio according to a preset frame length to obtain an audio frame set of the pre-communication audio; Extracting speech features of each audio frame included in the audio frame set based on Mel-frequency cepstral coefficients to obtain a speech feature set; Using the speech feature set as input data of a pre-trained deep learning-based speech recognition model, so that the deep learning-based speech recognition model outputs a text transcription result corresponding to each speech feature in the speech feature set, wherein the pre-trained deep learning-based speech recognition model is generated based on a pre-collected speech data training set; The text transcription result is traversed by using a named entity recognition model, and similarity matching is performed between the pre-built business keyword library and the word segmentation included in the text transcription result to obtain business keywords and business keyword annotations in the text transcription result; Processing the text transcription result using a trained intent classification model to obtain an intent classification result corresponding to the text transcription result, wherein the trained intent classification model is generated based on pre-collected business-related pre-communication text including intent category annotations; Determine the business type, business problem, and business-related information corresponding to the pre-communication audio according to the business keywords, the annotations of the business keywords, and the intention classification results; The business demand information of the target user is generated according to the business type, the business problem, and the business related information.

[0009] In a possible implementation of the first aspect, determining a target transaction corresponding to the business demand information according to the business demand information, and filling a business data pre-filled template of the target transaction according to the business demand information to generate a pre-filled business data document includes: Extracting one or more business elements from the business demand information according to the content included in the business demand information; According to a pre-built transaction mapping rule base including mapping relationships between business elements and transactions, and the business elements extracted from the business demand information, each business element is matched in sequence to obtain a target business element with the highest matching degree, and according to the target business element and the transaction mapping rule base, a target transaction corresponding to the target business element is obtained; Acquire the target field included in the business data pre-filled template of the target transaction; Identify the content included in the business requirement information according to the target field to obtain the target content corresponding to the target field; The target content in the business demand information is filled into the to-be-filled content corresponding to the target field in the business data pre-filled template to generate the pre-filled business data document.

[0010] In a possible implementation of the first aspect, the obtaining of the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage, wherein the video mode includes digital human video communication, include: Determining the type of the business requirement based on the business requirement information; Determining the selection priority of the candidate video agents in the formal communication stage according to the types of the business requirements and the scores of the candidate video agents corresponding to the types of the business requirements on the types of the business requirements; Determine, according to the selection priority, a video seat for the formal communication stage from the candidate video seats, and obtain an identifier of the video seat; Acquire the historical communication records of the target user, wherein the historical communication records include the historical communication video mode of the target user; If the proportion of the digital human video communication in the historical communication video mode is greater than or equal to a preset proportion threshold, it is determined that the video mode in the formal communication stage is the digital human video communication.

[0011] In a possible implementation of the first aspect, the identification of the video seat in the formal communication stage and the video mode in the formal communication stage, wherein the video mode includes digital human video communication, include: Acquire selection information of the pre-communication seat, wherein the selection information includes an identifier of the video seat in the formal communication stage and the video mode in the formal communication stage, and the identifier of the video seat includes an identifier of the pre-communication seat; According to the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage included in the selection information, the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage are acquired.

[0012] In combination with the second aspect of the present application, a deep learning-based video call system is provided, wherein the deep learning-based video call system includes a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions, and when the processor executes the machine-executable instructions, the deep learning-based video call system implements the aforementioned deep learning-based video call method.

[0013] In conjunction with the third aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed, the aforementioned deep learning-based video call method is implemented.

[0014] In conjunction with the fourth aspect of the present application, a computer program product is provided, and when the computer program is executed by a processor, the aforementioned deep learning-based video call method is implemented.

[0015] In combination with any of the above aspects, in the pre-communication stage, pre-communication information between the target user and the pre-communication seat and the user information of the target user are obtained, wherein the pre-communication information includes the pre-communication audio and the time of the pre-communication audio. According to the pre-communication audio, the business demand information of the target user is obtained. The identifier of the video seat in the formal communication stage and the video mode in the formal communication stage are obtained, wherein the video mode includes digital human video communication. According to the business demand information, the identifier of the video seat, the video mode, and the time of the pre-communication audio, a text message link is constructed, and the text message link is sent to the terminal device of the target user according to the user information of the target user, wherein the text message link includes a video call address for triggering a video call to the video seat. If a call request for a video call to the video seat is received, the video seat is called to establish a video call between the terminal device of the target user and the video seat in the formal communication stage. According to the business demand information, a target transaction corresponding to the business demand information is determined, and a business data pre-filled template of the target transaction is filled according to the business demand information to generate a pre-filled business data document. Based on the video mode and the pre-filled business data document, the formal communication interaction between the video agent and the target user is realized, so that by processing the pre-communication information, the key content of the pre-communication information is provided in advance to the video agent in the formal communication stage, thereby reducing the situation of repeated description, thereby improving the processing efficiency of business processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained by combining these drawings without paying creative work.

[0017] Figure 1 A flowchart of a deep learning-based video calling method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. According to the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] Figure 1 The flowchart of the deep learning-based video calling method provided by the embodiment of the present application is shown. It should be understood that in other embodiments, the order of some steps of the deep learning-based video calling method of this embodiment can be shared with each other according to actual needs, or some steps can be omitted or maintained. The video calling method based on deep learning includes: Step S110, obtaining pre-communication information between the target user and the pre-communication agent and user information of the target user in the pre-communication stage.

[0022] The pre-communication information includes the pre-communication audio and the time of the pre-communication audio.

[0023] In this embodiment, the server first needs to establish a communication connection with the pre-communication seat to receive the pre-communication audio data during the interaction between the pre-communication seat and the target user. This communication connection can be implemented based on a variety of network protocols, such as TCP / IP protocol, etc. When receiving the audio data, the server will also record the start time and end time of the received audio data as the time of the pre-communication audio. For the user information of the target user, the server can obtain it from a database that pre-stores user information. In this database, each user may correspond to a series of identification information, such as user name, user ID, and basic information filled in when the user registers (age, gender, contact information, etc.). When the target user starts to interact with the pre-communication seat, the server can query and obtain the corresponding user information from the database according to the identification of the target user (such as user ID).

[0024] Optionally, when obtaining the pre-communication audio, the server needs to ensure the integrity and accuracy of the audio data. For example, a cyclic redundancy check can be performed on the audio data to detect whether there is data corruption during transmission. At the same time, in order to adapt to different audio formats, the server can also integrate an audio format conversion function to convert the received audio data in various formats into a unified format for subsequent processing. For the time record of the pre-communication audio, the server can use its own system clock to accurately mark the start and end times of the audio data, and store this time information in association with the audio data.

[0025] For example, suppose that in a bank's customer service system, the server is responsible for handling the interaction between the customer (target user) and the pre-communication agent (bank customer service representative). When a customer calls the bank's customer service phone, the voice call will be transferred to the pre-communication agent. At this time, the server starts recording the conversation audio between the pre-communication agent and the customer. From the time the customer starts talking to the end of the conversation, the server accurately records this time period, for example, from 10:00:00 on August 1, 2023 to 10:05:00, which is the time of the pre-communication audio. At the same time, the server queries the customer's name, age, account type and other user information from the bank's user information database based on the customer's phone number (as the user ID).

[0026] Step S120: acquiring the business demand information of the target user according to the pre-communication audio.

[0027] In this embodiment, the server first pre-processes the pre-communication audio, for example, including the process of removing noise from the audio. The process can, for example, use a filtering algorithm in digital signal processing technology, such as a low-pass filter to remove high-frequency noise. Then, the audio is segmented according to certain time intervals or speech features, for example, segmented in units of sentences, and the entire audio is segmented into multiple speech segments. For each speech segment, the server uses speech recognition technology to convert the speech into text. This process involves the extraction of speech features, such as extracting speech features such as Mel-frequency cepstral coefficients, and then inputting these features into a pre-trained speech recognition model, and the model outputs the corresponding text content.

[0028] After obtaining the text content, the server performs semantic analysis on the text. First, a business keyword library is constructed, which contains keywords related to various banking businesses, such as "deposit", "loan", "transfer", etc. The server performs word segmentation on the text content, and then matches each word segmentation with the keywords in the business keyword library to determine the business keywords and their positions in the sentence (annotation). At the same time, the intent classification model is used to perform intent analysis on the entire text, such as determining whether it is to inquire about business, handle business, or complain about business. Finally, based on the business keywords, annotations, and intent classification results, the business demand information of the target user is comprehensively determined, including business type (such as loan business), business problems (such as loan interest rate inquiry), and business-related information (such as loan amount range, etc.).

[0029] For example, continuing with the above-mentioned bank customer service scenario, after the pre-communication audio is processed by the server, for example, a segment of the voice is converted into text "I want to know what the mortgage interest rate of your bank is". The server determines that "mortgage" is a business keyword and "what is the interest rate" is a business problem-related statement by matching with the business keyword library. The intent classification model determines that this is an intention to query the business. Combining this information, the server determines that the target user's business demand information is a business demand for querying mortgage interest rates, the business type is mortgage business, and the business problem is to query interest rates.

[0030] Step S130, obtaining the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage.

[0031] Among them, the video method includes digital human video communication.

[0032] In this embodiment, the server can first determine the type of business demand based on the business demand information that has been obtained. For example, if the business demand information indicates that it is a financial investment business, then the business demand type is financial investment. Then, the server queries the predefined business demand type and the scoring relationship table of candidate video agents. This table records the professional scores of each candidate video agent for different business demand types. For example, a video agent has a higher score in financial investment business, which may be because this agent has rich financial investment knowledge and experience. Based on these scores, the server determines the selection priority of candidate video agents in the formal communication stage.

[0033] To determine the video method, the server can obtain the historical communication records of the target user. This historical communication record is stored in the server's database, recording the video method used by the target user in previous video communications with the bank. If in these historical communication records, the proportion of real-person-driven digital human video communication is greater than or equal to the preset proportion threshold (such as 50%), then the video method in the formal communication stage is determined to be real-person-driven digital human video communication (that is, the digital human's movements and sounds are generated in real time based on the movements and sounds of the real video agent). If this condition is not met, other default video communication methods may be used, such as ordinary real-person video communication.

[0034] For example, in a banking customer service scenario, assume that the target user's business demand information is displayed as financial management business. The server queries the candidate video agent scoring table corresponding to the financial management business and finds that agent A has the highest score in the financial management business, followed by agent B. Then, according to the selection priority, agent A will be given priority as the video agent in the formal communication stage. For the video method, the server checks the historical communication records of the target user and finds that the target user has used real-person-driven digital human video communication in 6 of the past 10 video communications, accounting for 60%, which exceeds the preset 50% threshold. Therefore, the server determines that the video method in the formal communication stage is real-person-driven digital human video communication.

[0035] Step S140, constructing a text message link according to the business demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio, and sending the text message link to the terminal device of the target user according to the user information of the target user.

[0036] The SMS link includes a video call address for triggering a video call to the video agent.

[0037] In this embodiment, the server may first generate a primary key identifier for the SMS link based on the time of the pre-communication audio. This primary key identifier may be a certain encoding method based on time, such as converting the time into a string of a specific format, such as "20230801100000 - 20230801100500" (indicating the start and end time of the pre-communication audio), and then performing a hash operation to obtain a unique identifier. Alternatively, the start or end time of the pre-communication audio may be used to generate a primary key identifier using a rule such as year + month + day + hour + minute + second + random number. Among them, the random number may, for example, represent one or more of the business demand information, the identifier of the video seat, and the video mode. The primary key identifier, as well as the business demand information, the identifier of the video seat, the video mode, and other data corresponding to the primary key identifier may be stored in a database.

[0038] Next, the content of the SMS link and the video call address are generated based on the video agent's ID, video mode, and business demand information. The video call address may be a specific link generated based on the internal address of the video call service on the server plus the video agent's ID and other information. The content of the SMS link may include a brief description of the business demand (such as "financial management business consultation"), the video mode (such as "digital human video communication based on real-person drive"), and other information.

[0039] Finally, the SMS link's primary key identifier, content, and video call address are combined to construct a complete SMS link. The server uses the SMS gateway to send this SMS link to the target user's terminal device based on the target user's contact information (such as mobile phone number) in the user information. During the SMS sending process, the server needs to interact with the SMS gateway to transmit information such as the SMS content and the target mobile phone number according to the SMS gateway's interface specifications.

[0040] Step S150: If a call request for a video call to the video agent is received, the video agent is called to establish a video call between the terminal device of the target user and the video agent in the formal communication stage.

[0041] In this embodiment, after sending the SMS link, the server can start to monitor whether there is a video call request from the target user terminal device. The server can establish a special monitoring module, which can detect whether there is a request for a specific video call address based on a network communication protocol (such as HTTP protocol or a special video call protocol). When receiving a call request, the server first verifies the legitimacy of the request. This includes checking whether the request comes from the target user terminal device sent by the SMS link, which can be verified by verifying whether the device identifier (such as the IMEI code of the mobile phone or other unique identifier) ​​matches the previously stored target user device information.

[0042] If the request is legitimate, the server searches for the corresponding video agent resource based on the identity of the video agent. Then, the server sends a call notification to the video agent, notifying the video agent of a video call request from the target user. After receiving the notification, the video agent performs a series of signaling interactions with the server to establish a video call channel with the target user's terminal device.

[0043] Step S160, determining a target transaction corresponding to the business demand information according to the business demand information, and filling a business data pre-filled template of the target transaction according to the business demand information to generate a pre-filled business data document.

[0044] In this embodiment, the server may first extract one or more business elements from the business demand information. For example, if the business demand information is to query the mortgage interest rate, the business elements may include "mortgage", "interest rate", etc. Then, the server queries the pre-built transaction mapping rule library that includes the mapping relationship between business elements and transactions. This library defines the correspondence between different business elements and various transactions. For example, the "mortgage" business element may correspond to transactions such as "mortgage inquiry" and "mortgage application". The server matches the extracted business elements in turn, calculates the matching degree of each business element with the transaction in the library, and finds the target business element with the highest matching degree. According to the target business element and the transaction mapping rule library, the corresponding target transaction is determined, such as the "mortgage interest rate inquiry" transaction.

[0045] For the target transaction, the server obtains its business data pre-fill template. This template defines the various fields that need to be filled in for the transaction. For example, for a mortgage interest rate query transaction, there may be fields such as "loan amount range" and "loan term". The server identifies the content in the business demand information based on these target fields. For example, if the business demand information contains "I want to know the mortgage interest rate within 1 million and 30 years", then the server recognizes that "within 1 million" corresponds to the "loan amount range" field, and "30 years" corresponds to the "loan term" field. Finally, the target content in the business demand information is filled into the content to be filled in the corresponding field in the business data pre-fill template to generate a pre-filled business data document.

[0046] Step S170, realizing formal communication interaction between the video agent and the target user based on the video mode and the pre-filled business data document.

[0047] If the video method is based on real-person-driven digital human video communication, the server can first start the digital human-related services. This service includes digital human rendering, motion control, speech synthesis and other functions. The server passes the content in the pre-filled business data document to the digital human service, and the digital human can introduce the relevant business in the video communication based on this content. For example, if the pre-filled business data document contains information related to mortgage interest rates, the digital human can introduce the mortgage interest rate situation to the target user in the video with natural language and actions.

[0048] During the video communication process, the server is responsible for forwarding the video stream and data interaction. It forwards the video stream sent by the target user's terminal device to the video seat (here is the server corresponding to the digital human), and forwards the video stream generated by the digital human service to the target user's terminal device. The same is true for audio data, ensuring that both parties can communicate normally. If the target user has any new needs or feedback, the server will update the pre-filled business data document in real time and adjust the interactive content of the digital human according to the new situation.

[0049] Based on the above steps, in the pre-communication stage, the pre-communication information between the target user and the pre-communication seat and the user information of the target user are obtained, wherein the pre-communication information includes the pre-communication audio and the time of the pre-communication audio. According to the pre-communication audio, the business demand information of the target user is obtained. The identifier of the video seat in the formal communication stage and the video mode in the formal communication stage are obtained, wherein the video mode includes digital human video communication. According to the business demand information, the identifier of the video seat, the video mode, and the time of the pre-communication audio, a short message link is constructed, and the short message link is sent to the terminal device of the target user according to the user information of the target user, wherein the short message link includes a video call address for triggering a video call to the video seat. If a call request for a video call to the video seat is received, the video seat is called to establish a video call between the terminal device of the target user and the video seat in the formal communication stage. According to the business demand information, the target transaction corresponding to the business demand information is determined, and the business data pre-filled template of the target transaction is filled according to the business demand information to generate a pre-filled business data document. Based on the video mode and the pre-filled business data document, the formal communication interaction between the video agent and the target user is realized, so that by processing the pre-communication information, the key content of the pre-communication information is provided in advance to the video agent in the formal communication stage, thereby reducing the situation of repeated description, thereby improving the processing efficiency of business processing.

[0050] In a possible implementation, constructing a text message link according to the business demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio includes: A primary key identifier of the SMS link is generated according to the time of the pre-communication audio.

[0051] The content of the SMS link and the video call address are generated according to the identifier of the video agent, the video mode, and the service demand information.

[0052] The SMS link is constructed according to the primary key identifier of the SMS link, the content of the SMS link, and the video call address.

[0053] In this embodiment, after obtaining the time of the pre-communication audio, the server needs to convert it into a form suitable for use as a primary key identifier for a text message link. First, the server will accurately parse the start time and end time of the pre-communication audio. This involves processing the time format. For example, if the time is recorded in the format of "year-month-day hour: minute: second", the server needs to convert it into a unified, easy-to-process format, such as converting the date into a digital form, such as August 1, 2023 can be converted to 20230801. For the time part, a similar method is used for conversion, connecting the hours, minutes, and seconds, without using separators in the middle, for example, 10:05:00 is converted to 100500. Then, the start time and the end time are connected in a certain order to form a longer string, for example, "20230801100000 -20230801100500", which represents the pre-communication audio time range from 10:00:00 on August 1, 2023 to 10:05:00 on August 1, 2023.

[0054] Optionally, the server can encrypt or hash the time string after the connection to generate a unique primary key identifier. The hash algorithm can be selected from common algorithms such as MD5 or SHA-1. Taking the MD5 algorithm as an example, the server uses the time string as the input of the MD5 algorithm and obtains a 128-bit hash value after calculation. Since this hash value is usually expressed in hexadecimal, a 32-bit hexadecimal string is obtained. This string will be used as the primary key identifier of the SMS link. It has the characteristics of uniqueness and non-tamperability, which can ensure the unique identification of the SMS link in the entire system, and can also ensure the security of the data to a certain extent.

[0055] After generating the primary key identifier of the SMS link, the server can splice the identifier of the video seat, the video mode, the business demand information content, etc. to generate the content of the SMS link. And the video call address can be determined according to the identifier of the video seat and / or the video mode. For example, the video call address can be determined according to the identifier of the video seat, the video mode, and the preset video call address generation rules, or the mapping relationship between the preset video call address and the two. This address points to the video call entrance of a specific video seat. When the target user clicks the video call address in the SMS link, it can accurately connect to the corresponding video seat.

[0056] Finally, the primary key identifier of the SMS link, the content of the SMS link, and the video call address are concatenated to construct the SMS link.

[0057] In a possible implementation, it further includes: Get the preset call timeout.

[0058] The step of constructing a text message link according to the business demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio includes: A primary key identifier of the SMS link is generated according to the time of the pre-communication audio.

[0059] The content of the SMS link and the video call address are generated according to the identifier of the video agent, the video mode, and the service demand information.

[0060] The SMS link is constructed according to the primary key identifier of the SMS link, the content of the SMS link, the video call address, and the preset call time limit.

[0061] If a call request for a video call to the video agent is received, calling the video agent comprises: If a call request for a video call to the video agent is received within the preset call time limit, the video agent is called.

[0062] If a call request for a video call to the video agent is received outside the preset call time limit, the video agent is not called.

[0063] In this embodiment, the server obtains the preset call time efficiency, which is a process involving system configuration and data management. First, the server searches for settings related to the call time efficiency from its configuration file. This configuration file can be a file specifically used to store various system parameters and rules, such as in .ini or .xml format. In this configuration file, the preset call time efficiency may be defined as a specific time value, such as an integer in minutes.

[0064] In addition to obtaining it from the configuration file, the server may also obtain the preset call timeout from the database. In the database, there is a special table for storing the system's global setting information, which includes the preset call timeout parameter. This table structure may contain fields such as "setting name" (one of the records is "preset call timeout") and "setting value" (the corresponding time value). The server obtains the value of this parameter by executing an SQL query statement (for example, "SELECT setting value FROM system setting table WHERE setting name = 'preset call timeout'").

[0065] In addition, when obtaining the preset call time limit, the server can also consider various factors to ensure the rationality of the time limit setting. It will be adjusted in combination with business logic and user experience. For example, if the business involves urgent matters (such as video communication related to emergency fund transfer in financial transactions), the preset call time limit may be set shorter to ensure the timeliness of the business. For some relatively less urgent businesses (such as general product consultation), the call time limit may be relatively long. The server can also handle special needs in different regions or different user groups. For example, for users in different time zones, the server may dynamically adjust the preset call time limit according to the user's time zone. In the case of international business scenarios, the server will identify the user's time zone, and then adjust the preset call time limit according to factors such as the working hours and business busyness of the time zone. At the same time, for special user groups (such as VIP users), different preset call time limits may be provided. For example, the call time limit for VIP users may be longer than that for ordinary users to provide a better service experience.

[0066] After obtaining the preset call timeout value, the server will store this value in a specific variable in memory so that it can be quickly accessed when building SMS links and processing call requests. This variable will always exist during the server operation and can be dynamically updated based on system configuration updates (for example, the administrator modifies the preset call timeout value in the configuration file or database).

[0067] In the above implementation method of constructing an SMS link based on business demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio, an additional preset call time limit is spliced ​​into the SMS link to construct an SMS link including the preset call time limit, thereby reminding the customer that they need to call the video agent within the time limit.

[0068] When the server receives a video call request, it first obtains the current time. The server's system clock provides accurate current time information. Then, the server obtains the preset call validity period and the creation time of the SMS link from the SMS link (assuming that the server has parsed and stored the relevant information of the SMS link) (which can be inferred from the primary key identifier of the SMS link or other relevant information). At this time, the server will search for the corresponding video agent resource based on the identifier of the video agent. The server notifies the video agent of the video call request from the user by sending a call notification to this IP address and port number. After receiving the notification, the video agent will perform a series of signaling interactions with the server, such as sending confirmation information, negotiating the video encoding format, etc., and finally establish a video call with the user's terminal device.

[0069] If the current timestamp exceeds the time range of the SMS link creation time plus the preset call time limit, the server will directly reject the video call request. The server can send a prompt message to the user's terminal device to inform the user that the call time limit has expired and that the user needs to re-initiate the pre-communication process or contact customer service to obtain a new video call link. This is done to ensure the timeliness of the business process and the effective use of resources, avoiding the waste of video seat resources and confusion of business logic due to long waiting times.

[0070] In a possible implementation manner, obtaining the target user's business demand information according to the pre-communication audio includes: The pre-communication audio is framed according to a preset frame length to obtain an audio frame set of the pre-communication audio.

[0071] The speech feature of each audio frame included in the audio frame set is extracted based on the Mel-frequency cepstral coefficient to obtain a speech feature set.

[0072] The speech feature set is used as input data for a pre-trained deep learning-based speech recognition model, so that the deep learning-based speech recognition model outputs a text transcription result corresponding to each speech feature in the speech feature set, wherein the pre-trained deep learning-based speech recognition model is generated based on a pre-collected speech data training set.

[0073] The text transcription result is traversed by using a named entity recognition model, and similarity matching is performed between the pre-built business keyword library and the word segmentation included in the text transcription result to obtain the business keywords and business keyword annotations in the text transcription result.

[0074] The text transcription result is processed using a trained intent classification model to obtain an intent classification result corresponding to the text transcription result, wherein the trained intent classification model is generated based on pre-collected business-related pre-communication text training including intent category annotations.

[0075] The business type, business problem, and business-related information corresponding to the pre-communication audio are determined based on the business keywords, the annotations of the business keywords, and the intention classification results.

[0076] The business demand information of the target user is generated according to the business type, the business problem, and the business related information.

[0077] In this embodiment, when the server performs frame processing on the pre-communication audio, it is first necessary to determine the preset frame length. This preset frame length is a fixed duration determined based on the experience and requirements of audio processing, for example, it can be set to 20 milliseconds. After determining the frame length, the server starts from the starting point of the pre-communication audio and divides the audio into multiple segments according to this fixed frame length, and each segment is an audio frame. In the frame division process, the server needs to deal with issues such as the format and sampling rate of the audio data. Assuming that the pre-communication audio is stored in a certain standard audio format (such as WAV format), the server will first parse the audio file header to obtain parameter information such as the sampling rate of the audio (for example, 44.1kHz). Then, the number of sampling points contained in each frame is calculated based on the sampling rate and frame length. For example, for a sampling rate of 44.1kHz and a frame length of 20 milliseconds, the number of sampling points contained in each frame is 44100 * 0.02 = 882 sampling points. The server extracts audio frames from the audio data in turn according to the calculated number of sampling points, and combines these audio frames to form an audio frame set of the pre-communication audio. This process requires precise calculation and data processing to ensure the integrity and accuracy of each audio frame. At the same time, for the convenience of subsequent processing, the server may number each audio frame, starting from 1 and increasing in sequence, so that each audio frame can be easily indexed and operated.

[0078] For each audio frame in the audio frame set, the server extracts the Mel frequency cepstral coefficient as a speech feature. First, the server performs pre-emphasis processing on each audio frame, which helps to improve the spectrum of the high-frequency part and improve the distinguishability of the speech features. Then, the server performs frame segmentation and windowing processing on the pre-emphasized audio frame. Framing is to further divide the audio frame into smaller segments, and windowing is to reduce spectrum leakage. Commonly used window functions include Hamming window, which implements the windowing operation by multiplying each sample value of the audio frame by the value of the Hamming window function. Then, the server can perform a fast Fourier transform on the windowed audio frame to convert the time domain signal into a frequency domain signal. After obtaining the frequency domain signal, the energy of the Mel filter bank is calculated. The Mel filter bank is a set of filters uniformly distributed on the Mel scale. The frequency domain signal is passed through these filters and the output energy of each filter is calculated. Finally, the energy of the Mel filter bank is logarithmic and discrete cosine transformed, and the result is the Mel frequency cepstral coefficient. The server performs the above operation on each audio frame in the audio frame set, and combines the Mel-frequency cepstral coefficients extracted from each audio frame to form a speech feature set.

[0079] The server inputs the previously obtained speech feature set into a pre-trained deep learning-based speech recognition model. This speech recognition model is built based on a large number of pre-collected speech data training sets. During the training process, the speech data training set contains a variety of different speech samples, covering different speech content, accents, speaking speed, etc. The model structure may use a deep neural network (such as a convolutional neural network, a long short-term memory network, or a combination of the two). During training, the speech samples are converted into speech features as input, and the corresponding text transcription results are used as output. The weights of the model are continuously adjusted through the back propagation algorithm so that the model can accurately map the input speech features to the correct text. When the server inputs the current speech feature set into this trained model, the neurons inside the model calculate the input speech features according to the predefined network structure and weights. For example, in a convolutional neural network, the convolution layer will perform convolution operations on the speech features to extract local features; the pooling layer will downsample the features to reduce the amount of data while retaining key features; the fully connected layer will integrate the previous features and finally output the text transcription results corresponding to each speech feature. This result is a text representation of the speech content in the audio frame.

[0080] After the server obtains the text transcription result, it can use the named entity recognition model to process it. First, the model will perform word segmentation on the text transcription result and decompose the entire sentence into individual words. For example, for the text transcription result "I want to check the loan interest of the bank", it will be segmented into "I", "I want", "query", "a while", "bank", "of", "loan", and "interest". Then, the server will match these segmented words with the pre-built business keyword library for similarity. The business keyword library is a collection of various keywords related to the business. For example, in the financial business scenario, it may contain keywords such as "deposit", "loan", "interest rate", and "transfer". For each segmented word, the server will calculate its similarity with each keyword in the business keyword library. The similarity can be calculated using a variety of methods, such as cosine similarity, edit distance, etc. When a business keyword with a similarity exceeding a certain threshold (such as 0.8) is found, the segmented word is marked as a business keyword. At the same time, in order to clarify the position and role of the business keyword in the sentence, it will also be labeled. For example, for the business keyword "loan", the label may be "noun, as a query object". By traversing and matching the entire text transcription results, the server can determine the business keywords and their annotations.

[0081] The server inputs the text transcription result into the trained intent classification model. This intent classification model is built based on the pre-collected business-related pre-communication texts. When building the training set, these pre-communication texts are labeled with intent categories, such as "query category", "processing category", "complaint category", etc. The structure of the intent classification model can be based on machine learning algorithms (such as support vector machines, decision trees, etc.) or deep learning algorithms (such as neural networks). If it is a neural network model, during the training process, the pre-communication text is converted into a suitable feature representation (such as word vector) as input, and the corresponding intent category label is used as output. By continuously adjusting the model weight, the model can accurately classify the input text intent. When the server inputs the current text transcription result into this trained model, the model will analyze the text transcription result according to the internal calculation logic. For example, in the neural network model, through the calculation of multiple layers of neurons, the intent classification result corresponding to the text transcription result is finally output. For example, if the text transcription result is "I want to inquire about the loan interest of the bank", the intent classification model may output the intent classification result of "query category".

[0082] The server determines the business type, business problem, and business-related information corresponding to the pre-communication audio based on the business keywords, business keyword annotations, and intent classification results obtained previously. For the determination of the business type, if the business keyword contains "loan" and the intent classification result is "query type", then the business type may be determined as "loan query business". The determination of business problems is more dependent on the annotation of business keywords. For example, the business keyword "loan interest" is annotated as "query object", then the business problem is "query loan interest". Business-related information may be obtained from other parts of the text transcription result. For example, the text transcription result is "I want to inquire about the interest on the bank's 1 million loan", where "1 million" is business-related information, indicating the loan amount.

[0083] The server generates the target user's business demand information based on the determined business type, business problem, and business-related information. Business demand information is a comprehensive description of the target user's needs. For example, if the business type is "loan inquiry business", the business problem is "inquiry of loan interest", and the business-related information is "loan amount 1 million", then the generated business demand information may be "inquiry of the interest of a 1 million loan (loan inquiry business)". This business demand information can accurately reflect the target user's needs in the pre-communication audio, and provides an important basis for subsequent video seat arrangement, business processing and other processes.

[0084] In a possible implementation, determining a target transaction corresponding to the business demand information according to the business demand information, and filling a business data pre-filled template of the target transaction according to the business demand information to generate a pre-filled business data document includes: One or more business elements are extracted from the business demand information according to the content included in the business demand information.

[0085] According to a pre-constructed transaction mapping rule base including a mapping relationship between business elements and transactions, and the business elements extracted from the business demand information, each business element is matched in turn to obtain a target business element with the highest matching degree, and according to the target business element and the transaction mapping rule base, a target transaction corresponding to the target business element is obtained.

[0086] Obtain the target fields included in the business data pre-filled template of the target transaction.

[0087] The content included in the business requirement information is identified according to the target field to obtain the target content corresponding to the target field.

[0088] The target content in the business demand information is filled into the to-be-filled content corresponding to the target field in the business data pre-filled template to generate the pre-filled business data document.

[0089] In this embodiment, the server first performs a detailed analysis of the business demand information. The business demand information is a comprehensive description of user needs, which may be a natural language text or data organized in a specific format. The server extracts business elements through natural language processing technology or according to established parsing rules. For example, the business demand information is "I want to inquire about the after-sales service of electronic products, especially the warranty period and repair location". The server will identify "electronic products", "after-sales service", "warranty period", "repair location" and so on as business elements. This process may involve lexical analysis, that is, breaking down the text into words or phrases, and then filtering out the parts with business significance as business elements according to business-related semantic rules. The server will build a temporary data structure to store these extracted business elements for subsequent processing.

[0090] The transaction mapping rule base in the server is a pre-built knowledge base that stores the mapping relationship between business elements and various transactions. For example, in a scenario of electronic product sales and services, "electronic products" and "after-sales service" may be associated with "after-sales query transaction"; "electronic products" and "warranty period" may be associated with "warranty terms query transaction". The server matches each business element extracted from the business demand information with the records in the transaction mapping rule base one by one. For each business element, calculate its matching degree with each record in the library. The matching degree can be calculated based on a variety of methods, such as the number of keyword matches, semantic similarity, etc. For example, if the business elements are "electronic products" and "warranty period", there is a record in the transaction mapping rule base called "electronic product warranty period query-warranty terms query transaction", which contains the keywords in these two business elements, then the matching degree between this business element and this record is high. The server calculates the matching degree of all business elements and finds the business element combination with the highest matching degree as the target business element. According to this target business element, find the corresponding transaction in the transaction mapping rule base, and this transaction is the target transaction. For example, if the target business elements are "electronic products" and "warranty period", the corresponding target transaction can be "warranty terms query transaction".

[0091] After determining the target transaction, the server obtains the business data pre-filled template of the target transaction from the database storing the template. This template is pre-designed according to the characteristics of the target transaction and defines various information fields required to complete the transaction.

[0092] The server performs targeted identification on the content in the business demand information based on the target field obtained previously. For example, the target field is "product model" and the business demand information is "I purchased an electronic product of model YY of brand XX and want to check the warranty period". The server will search for content related to "product model" in this business demand information, that is, "YY model". This "YY model" is the target content corresponding to the target field "product model". This identification process can be achieved through text processing techniques, such as pattern matching and keyword positioning. The server traverses each part of the business demand information and compares it with the target field. When matching or related content is found, it is identified as the target content and recorded and associated with the corresponding target field.

[0093] After obtaining the target field and the corresponding target content, the server fills the target content into the corresponding position in the business data pre-fill template. For example, in the content to be filled in the "Product Model" field in the business data pre-fill template, the server fills in the previously identified "YY Model". For each target field and its corresponding target content, the filling operation is performed in this way. When all the target content is filled, a pre-filled business data document is generated. This pre-filled business data document contains some known information related to the target transaction, providing a preliminary data basis for subsequent business processing, which can improve the efficiency and accuracy of business processing.

[0094] In a possible implementation, the obtaining of the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage, wherein the video mode includes digital human video communication, include: The type of the business requirement is determined based on the business requirement information.

[0095] The selection priority of the candidate video agents in the formal communication stage is determined according to the types of the business requirements and the scores of the candidate video agents corresponding to the types of the business requirements for the types of the business requirements.

[0096] According to the selection priority, a video seat for the formal communication stage is determined from the candidate video seats, and an identifier of the video seat is obtained.

[0097] The historical communication records of the target user are obtained, wherein the historical communication records include the historical communication video modes of the target user.

[0098] If the proportion of the digital human video communication in the historical communication video mode is greater than or equal to a preset proportion threshold, it is determined that the video mode in the formal communication stage is the digital human video communication.

[0099] In this embodiment, the server can first conduct an in-depth analysis of the business demand information. The business demand information is a comprehensive description of the target user's needs, including various business-related contents. The server will use natural language processing technology or predefined business classification rules to determine the type of business demand. For example, the business demand information is "query the maintenance process of a certain electronic product". The server determines the type of business demand as "product maintenance query class" through the keywords "query", "maintenance process" and the understanding of the "electronic product" business field. If the business demand information is "request to update the version of a certain software", the server will determine the type of business demand as "software update request class" based on keywords such as "request" and "software version update". The server can pre-build a business demand type classification model or rule library, which contains various business demand information patterns and corresponding types. When analyzing the business demand information, it will be matched with the content in the model or library to determine the type of business demand.

[0100] The server stores the scoring information of each candidate video agent for different types of business needs. These scores are obtained based on the pre-assessment of the agent's expertise, experience, and business capabilities. For example, for the business need type of "product maintenance inquiry", the score of candidate video agent A is 80 points, the score of agent B is 70 points, and the score of agent C is 75 points. The server determines the selection priority based on these scores. In this example, agent A has the highest score, so agent A has the highest selection priority for the business need of "product maintenance inquiry", followed by agent C, and finally agent B. Among them, the server can maintain a scoring matrix or database table, in which the rows represent candidate video agents, the columns represent different types of business needs, and each intersection stores the corresponding score. After determining the type of business need, the server obtains the score of each candidate video agent by querying this matrix or table, and then determines the selection priority.

[0101] According to the previously determined selection priority, the server selects the agent with the highest priority from the candidate video agents as the video agent in the formal communication stage. For example, in the "product maintenance inquiry" business requirement mentioned above, agent A has the highest selection priority, so the server determines agent A as the video agent in the formal communication stage.

[0102] The server can obtain the historical communication records of the target user from the database that stores user data. This database is specifically used to store various user interaction data, including historical communication records. In the historical communication records, each record contains information such as the time of communication, the summary of the communication content, and the video method used for communication. The server can retrieve the historical communication records related to the user from the database by querying the target user's identifier (such as user ID).

[0103] After obtaining information related to the target user's historical communication video methods, the server calculates the proportion of digital human video communication in all historical communication video methods. Assume that the preset proportion threshold is 50%. The server will count the number of times digital human video communication appears in the historical communication records. Assuming that there are a total of 10 historical communication records, of which digital human video communication appears 6 times, then the proportion of digital human video communication is 60%, which is greater than the preset 50% proportion threshold. If this proportion is greater than or equal to the preset proportion threshold, the server determines that the video method in the formal communication stage is digital human video communication. If the proportion is less than the threshold, the video method may be determined based on other rules or default settings, such as using ordinary video communication methods or recommending specific video methods according to business demand types.

[0104] In a possible implementation, the identification of the video seat in the formal communication stage and the video mode in the formal communication stage, wherein the video mode includes digital human video communication, include: The selection information of the pre-communication seat is obtained, wherein the selection information includes the identifier of the video seat in the formal communication stage and the video mode in the formal communication stage, and the identifier of the video seat includes the identifier of the pre-communication seat.

[0105] According to the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage included in the selection information, the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage are acquired.

[0106] In this embodiment, in the pre-communication stage, the pre-communication agent can manually select the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage. The identifier of the video agent includes the identifier of the pre-communication agent, that is, the pre-communication agent can choose whether to answer the video call by himself or herself in the formal communication stage, or choose to have other video agents answer the call. According to the pre-communication agent manually selecting the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage, it is stored as the selection information of the pre-communication agent.

[0107] The server can obtain the selection information of the pre-communication seat to obtain the identifier of the video seat in the formal communication stage and the video mode in the formal communication stage according to the identifier of the video seat in the formal communication stage and the video mode in the formal communication stage included in the selection information.

[0108] In the above embodiments, the deep learning-based video call system for executing the above method embodiments has at least one processor, a control module (chip set) coupled to at least one of the (at least one) processors, a memory coupled to the control module, a non-volatile memory (NVM) / storage device coupled to the control module, at least one input / output device coupled to the control module, and a network interface coupled to the control module.

[0109] The processor may include at least one single-core or multi-core processor, and the processor may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). For some alternative implementations, the deep learning-based video call system can be used as a deep learning-based video call system device such as the gateway described in the embodiments of the present application.

[0110] For some alternative embodiments, the deep learning based video calling system may include at least one computer readable medium (e.g., a memory or NVM / storage device) having instructions and at least one processor integrated with the at least one computer readable medium and configured to execute the instructions to implement a module to perform the actions described in the present disclosure.

[0111] For one embodiment, the control module may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) and / or any suitable device or component in communication with the control module.

[0112] The control module may include a memory controller module to provide an interface to the memory. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0113] The memory may be used, for example, to load and store data and / or instructions for a deep learning based video calling system. For one embodiment, the memory may include any suitable volatile memory, for example, a suitable DRAM.

[0114] For one embodiment, the control module may include at least one input / output controller to provide an interface to the NVM / storage device and the (at least one) input / output device.

[0115] For example, the NVM / storage device may be used to store data and / or instructions. The NVM / storage device may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (at least one) non-volatile storage device (e.g., at least one hard disk drive (HDD), at least one compact disk (CD) drive, and / or at least one digital versatile disk (DVD) drive).

[0116] The NVM / storage device may include storage resources that are physically part of the device on which the deep learning based video calling system is installed, or it may be accessible to the device without being part of the device. For example, the NVM / storage device may be accessed via (at least one) input / output device over a network.

[0117] The (at least one) input / output device may provide an interface for the deep learning-based video call system to communicate with any other appropriate device, and the input / output device may include a communication component, a phonetic component, a sensor component, etc. The network interface may provide an interface for the deep learning-based video call system to communicate based on at least one network, and the deep learning-based video call system may wirelessly communicate with at least one component of a wireless network based on any standard and / or protocol of at least one wireless network standard and / or protocol, such as accessing a wireless network based on a communication standard.

[0118] For one embodiment, at least one of the (at least one) processor may be loaded together with the logic of at least one controller of the control module (e.g., a memory controller module). For one embodiment, at least one of the (at least one) processor may be loaded together with the logic of at least one controller of the control module to form a system level load. For one embodiment, at least one of the (at least one) processor may be integrated on the same die with the logic of at least one controller of the control module. For one embodiment, at least one of the (at least one) processor may be integrated on the same die with the logic of at least one controller of the control module to form a system on chip (SoC).

[0119] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0120] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the deep learning-based video call method described in the aforementioned embodiment.

[0121] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the deep learning-based video call method described in the aforementioned embodiment.

[0122] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0123] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other medium that can be used to be computer-readable with or store data.

[0124] Finally, it should be noted that what is disclosed above is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A video calling method based on deep learning, characterized in that: include: In the pre-communication stage, pre-communication information between the target user and the pre-communication agent and user information of the target user are obtained, wherein the pre-communication information includes pre-communication audio and the time of the pre-communication audio; Acquiring the business demand information of the target user according to the pre-communication audio; Obtaining the identification of the video seat in the formal communication stage, and the video mode in the formal communication stage, wherein the video mode includes digital human video communication; Constructing a text message link according to the service demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio, and sending the text message link to the terminal device of the target user according to the user information of the target user, wherein the text message link includes a video call address for triggering a video call to the video agent; If a call request for a video call to the video agent is received, calling the video agent to establish a video call between the terminal device of the target user and the video agent in the formal communication stage; Determine a target transaction corresponding to the business demand information according to the business demand information, and fill in a business data pre-filled template of the target transaction according to the business demand information to generate a pre-filled business data document; Based on the video mode and the pre-filled business data document, formal communication interaction is achieved between the video agent and the target user.

2. The deep learning-based video calling method according to claim 1, characterized in that: The step of constructing a text message link according to the business demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio includes: Generate a primary key identifier of the SMS link according to the time of the pre-communication audio; Generate the content of the SMS link and the video call address according to the identifier of the video agent, the video mode, and the service demand information; The SMS link is constructed according to the primary key identifier of the SMS link, the content of the SMS link, and the video call address.

3. The deep learning-based video calling method according to claim 2, characterized in that: Also includes: Get the preset call time limit; The step of constructing a text message link according to the business demand information, the identifier of the video agent, the video mode, and the time of the pre-communication audio includes: Generate a primary key identifier of the SMS link according to the time of the pre-communication audio; Generate the content of the SMS link and the video call address according to the identifier of the video agent, the video mode, and the service demand information; Constructing the SMS link according to the primary key identifier of the SMS link, the content of the SMS link, the video call address, and the preset call time limit; If a call request for a video call to the video agent is received, calling the video agent comprises: If a call request for a video call to the video agent is received within the preset call time limit, calling the video agent; If a call request for a video call to the video agent is received outside the preset call time limit, the video agent is not called.

4. The deep learning-based video calling method according to claim 3, characterized in that: The acquiring the business demand information of the target user according to the pre-communication audio includes: Performing frame processing on the pre-communication audio according to a preset frame length to obtain an audio frame set of the pre-communication audio; Extracting speech features of each audio frame included in the audio frame set based on Mel-frequency cepstral coefficients to obtain a speech feature set; Using the speech feature set as input data of a pre-trained deep learning-based speech recognition model, so that the deep learning-based speech recognition model outputs a text transcription result corresponding to each speech feature in the speech feature set, wherein the pre-trained deep learning-based speech recognition model is generated based on a pre-collected speech data training set; The text transcription result is traversed by using a named entity recognition model, and similarity matching is performed between the pre-built business keyword library and the word segmentation included in the text transcription result to obtain business keywords and business keyword annotations in the text transcription result; Processing the text transcription result using a trained intent classification model to obtain an intent classification result corresponding to the text transcription result, wherein the trained intent classification model is generated based on pre-collected business-related pre-communication text including intent category annotations; Determine the business type, business problem, and business-related information corresponding to the pre-communication audio according to the business keywords, the annotations of the business keywords, and the intention classification results; The business demand information of the target user is generated according to the business type, the business problem, and the business related information.

5. The deep learning-based video calling method according to claim 4, characterized in that: The step of determining a target transaction corresponding to the business demand information according to the business demand information, and filling a business data pre-filled template of the target transaction according to the business demand information to generate a pre-filled business data document includes: Extracting one or more business elements from the business demand information according to the content included in the business demand information; According to a pre-built transaction mapping rule base including mapping relationships between business elements and transactions, and the business elements extracted from the business demand information, each business element is matched in sequence to obtain a target business element with the highest matching degree, and according to the target business element and the transaction mapping rule base, a target transaction corresponding to the target business element is obtained; Acquire the target field included in the business data pre-filled template of the target transaction; Identify the content included in the business requirement information according to the target field to obtain the target content corresponding to the target field; The target content in the business demand information is filled into the to-be-filled content corresponding to the target field in the business data pre-filled template to generate the pre-filled business data document.

6. The deep learning-based video calling method according to any one of claims 1 to 5, characterized in that: The step of obtaining the identification of the video seat in the formal communication stage and the video mode in the formal communication stage, wherein the video mode includes digital human video communication, comprises: Determining the type of the business requirement based on the business requirement information; Determining the selection priority of the candidate video agents in the formal communication stage according to the types of the business requirements and the scores of the candidate video agents corresponding to the types of the business requirements on the types of the business requirements; Determine, according to the selection priority, a video seat for the formal communication stage from the candidate video seats, and obtain an identifier of the video seat; Acquire the historical communication records of the target user, wherein the historical communication records include the historical communication video mode of the target user; If the proportion of the digital human video communication in the historical communication video mode is greater than or equal to a preset proportion threshold, it is determined that the video mode in the formal communication stage is the digital human video communication.

7. The deep learning-based video calling method according to any one of claims 1 to 5, characterized in that: The identification of the video seat in the formal communication stage is obtained, and the video mode in the formal communication stage includes digital human video communication, including Acquire selection information of the pre-communication seat, wherein the selection information includes an identifier of the video seat in the formal communication stage and the video mode in the formal communication stage, and the identifier of the video seat includes an identifier of the pre-communication seat; According to the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage included in the selection information, the identifier of the video agent in the formal communication stage and the video mode in the formal communication stage are acquired.

8. A video calling system based on deep learning, characterized in that: It includes a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by a computer, the deep learning-based video calling method described in any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by a computer, the deep learning-based video calling method described in any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images