Method and device for collaborative interaction of voice and text in a call, equipment and medium

Abstract

The application relates to the technical field of voice interaction, and discloses a voice and text collaborative interaction method, device and equipment in a call and a medium, which comprises the following steps: establishing a voice call session between a terminal device and a voice interaction node, and establishing a text interaction session when permission verification is passed; receiving to-be-verified text information through a visual interaction interface, generating a digital reference identifier, text mapping records and a digital input message; after the digital input message reaches a session control node, reading the text mapping records and extracting the to-be-verified text information, and generating an information verification result. The application can be applied to business scenes such as financial technology and medical health, to-be-verified text information is received through a text interaction session, and text information feedback and reading are completed through a digital reference identifier and text mapping records, so that accurate text information can enter an information verification process in combination with voice recognition text, and recognition deviation caused by pure dependence on voice recognition is reduced.

Description

Technical Field

[0001] This invention relates to the field of voice interaction technology, and in particular to a method, apparatus, device, and medium for collaborative voice and text interaction during a call. Background Technology

[0002] Existing call center services typically rely on intelligent speech recognition and natural language processing technologies to identify customer intent, guide customers through processes, and collect information. For general inquiries, voice interaction can meet basic service needs; however, in scenarios requiring precise text information such as names, addresses, institution names, account names, identification information, and patient information, speech-recognized text is easily affected by dialects, accents, homophones, proper nouns, and environmental noise, making it difficult to reliably meet the information verification requirements of the current business stage. Traditional telephone keypad input is also ill-suited for handling Chinese characters and long texts, resulting in inaccurate text information being reliably collected during voice calls and unable to proceed to subsequent verification processes.

[0003] In the fintech sector, telephone customer service is commonly used for scenarios such as bank account opening inquiries, account information confirmation, payment verification, loan applications, credit card services, securities business inquiries, insurance policy inquiries, and wealth management product services. These scenarios frequently require customers to provide their name, account name, account opening address, payee information, policy information, transaction purpose, and identity verification information. Existing voice customer service systems are prone to errors in recognizing this precise text information due to pronunciation differences, dialects, homophones, and similar financial product names, making it difficult for current business nodes to complete information verification based on the recognized text.

[0004] In the healthcare sector, telephone customer service is commonly used for scenarios such as appointment booking, report inquiries, medical insurance consultations, patient information confirmation, medication delivery address confirmation, and health management follow-ups. These scenarios frequently require the collection of precise textual information such as the patient's name, hospital department name, medication name, medical insurance information, and residential address. Current voice interaction methods often suffer from discrepancies between the speech-recognized text and the customer's actual expression when handling medical terminology, medication names, regional accents, and long address information. This makes it difficult to obtain accurate textual information for verification at current business nodes. Summary of the Invention

[0005] The main objective of this invention is to provide a method, apparatus, device, and storage medium for collaborative interaction between voice and text during a call. This invention aims to solve the technical problem that existing technologies, when relying on voice recognition or telephone keypad input to collect accurate text information in voice call scenarios, lack a data flow mechanism to associate text information with the voice call session and the current business node and transmit it to the session control side for information verification.

[0006] To achieve the above objectives, the present invention provides a method for collaborative voice and text interaction during a phone call, comprising: The system receives a voice call request sent by a terminal device, causes the voice call request to reach the session control node, extracts terminal device information and called number from the voice call request, establishes a voice call session between the terminal device and the voice interaction node based on the called number, and generates a call session identifier corresponding to the voice call session. After the voice call session is established, text interaction permission is verified based on the call session identifier and the terminal device information. When the text interaction permission verification is passed, a text interaction session bound to the voice call session is established. Based on the text interaction session, an interface creation request is sent to the terminal device, so that the terminal device can display a visual interactive interface while maintaining the voice call session. Based on the voice call session, speech recognition text is generated, and the current business node and text input task are determined according to the speech recognition text; Based on the text input task, the system receives text information to be verified through the visual interactive interface, generates a digital reference identifier bound to the text information to be verified, writes the digital reference identifier, the text information to be verified, and the call session identifier into the text mapping cache to generate a text mapping record, generates a digital input message based on the digital reference identifier, and sends the digital input message to the session control node. After the numeric input message arrives at the session control node, the numeric reference identifier is obtained based on the numeric input message. The text mapping record is read from the text mapping cache according to the call session identifier and the numeric reference identifier. The text information to be verified is extracted from the text mapping record. An information verification result is generated based on the current service node, the speech recognition text, and the text information to be verified.

[0007] Furthermore, to achieve the above objectives, the present invention provides a voice and text collaborative interaction device during a call, comprising: The call session establishment module is used to receive a voice call request sent by a terminal device, make the voice call request reach the session control node, extract terminal device information and called number from the voice call request, establish a voice call session between the terminal device and the voice interaction node based on the called number, and generate a call session identifier corresponding to the voice call session. The text permission verification module is used to verify text interaction permissions based on the call session identifier and the terminal device information after the voice call session is established, and to establish a text interaction session bound to the voice call session when the text interaction permission verification is passed. The visual interface push module is used to send an interface creation request to the terminal device based on the text interaction session, so that the terminal device can display a visual interactive interface while maintaining the voice call session. The speech semantic recognition module is used to generate speech recognition text based on the voice call session, and determine the current business node and text input task based on the speech recognition text; The text mapping forwarding module is used to receive text information to be verified through the visual interactive interface based on the text input task, generate a digital reference identifier bound to the text information to be verified, write the digital reference identifier, the text information to be verified and the call session identifier into the text mapping cache to generate a text mapping record, generate a digital input message based on the digital reference identifier, and send the digital input message to the session control node. The text information verification module is used to obtain the digital reference identifier based on the digital input message after the digital input message arrives at the session control node, read the text mapping record from the text mapping cache according to the call session identifier and the digital reference identifier, extract the text information to be verified from the text mapping record, and generate an information verification result based on the current service node, the speech recognition text and the text information to be verified.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a voice and text collaborative interaction program stored in the memory and executable on the processor, wherein when the voice and text collaborative interaction program is executed by the processor, it implements the steps of the voice and text collaborative interaction method described above.

[0009] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a voice and text collaborative interaction program during a call, wherein when the voice and text collaborative interaction program during a call is executed by a processor, it implements the steps of the voice and text collaborative interaction method during a call as described above.

[0010] Beneficial Effects: This invention relates to the field of voice interaction technology, and discloses a method, apparatus, device, and medium for collaborative voice and text interaction during a call. The method includes: receiving a voice call request from a terminal device; establishing a voice call session between the terminal device and a voice interaction node; generating a call session identifier; performing text interaction permission verification after the voice call session is established, and establishing a text interaction session upon successful verification; sending an interface establishment request to the terminal device based on the text interaction session, enabling the terminal device to display a visual interactive interface; generating speech-recognized text based on the voice call session, and determining the current business node and text input task; receiving text information to be verified through the visual interactive interface, generating a numeric reference identifier, a text mapping record, and a numeric input message; and reading the text mapping record and extracting the text information to be verified after the numeric input message arrives at the session control node, generating an information verification result. This invention can be applied to business scenarios such as fintech and healthcare. By establishing a text interaction session in a voice call, the text information to be verified is converted into a digital reference identifier and a digital input message. This allows the text information to be verified to correspond to the call session identifier through a text mapping cache and be read and verified at the session control node. Since the information verification result is generated by combining the current business node, the voice-recognized text, and the text information to be verified, the recognition error when accurate text information relies solely on voice recognition is reduced. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of an application environment for a voice and text collaborative interaction method during a call, according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating an embodiment of the voice and text collaborative interaction method during a call according to the present invention; Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the voice and text collaborative interaction device in a call according to the present invention; Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention; Figure 6 This is a schematic diagram of the system architecture of a voice and text collaborative interaction method in a call according to an embodiment of the present invention. Detailed Implementation

[0012] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0013] The voice and text collaborative interaction method provided in this invention can be applied to, for example...Figure 1 In this application environment, the client communicates with the server via a network. The server can receive voice call requests from terminal devices through the client, establish a voice call session between the terminal device and the voice interaction node, and generate a call session identifier. After the voice call session is established, it performs text interaction permission verification, and establishes a text interaction session if successful. Based on the text interaction session, it sends an interface establishment request to the terminal device, enabling the terminal device to display a visual interactive interface. Based on the voice call session, it generates speech recognition text and determines the current business node and text input task. It receives the text information to be verified through the visual interactive interface, generates a numeric reference identifier, a text mapping record, and a numeric input message. After the numeric input message arrives at the session control node, it reads the text mapping record, extracts the text information to be verified, and generates an information verification result. This invention can be applied to business scenarios such as fintech and healthcare. It establishes a text interaction session within a voice call, converting the text information to be verified into a numerical reference identifier and a numerical input message. This allows the text information to be mapped to the call session identifier via a text mapping cache and then read and verified at the session control node. Because the verification result is generated by combining the current business node, the speech-recognized text, and the text information to be verified, the recognition bias is reduced when accurate text information relies solely on speech recognition. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0014] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the voice and text collaborative interaction method during a call provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0015] like Figure 2 As shown, the voice and text collaborative interaction method in a call proposed in this invention includes the following steps: S10, receive a voice call request sent by a terminal device, make the voice call request reach the session control node, extract terminal device information and called number from the voice call request, establish a voice call session between the terminal device and the voice interaction node based on the called number, and generate a call session identifier corresponding to the voice call session. In this embodiment, the terminal device can be a mobile terminal, a fixed-line terminal, or a service terminal with voice call capabilities. The voice call request is initiated by the terminal device and may carry the calling party identifier, the called number, the terminal address, the access network type, and a call timestamp. Upon receiving the voice call request, the access side verifies the number format and terminal source, and retains data that forms the terminal device information and the called number.

[0016] The session control node is used to receive voice call requests and control call allocation. It can be implemented by a session control server, call dispatch service, or cloud communication session control unit. When a voice call request reaches the session control node, it can be encapsulated as call access signaling, which includes terminal device information and the called number. The terminal device information can consist of a terminal identifier, terminal address, calling number, and access network type. The called number is used to identify the service entry point, such as an account service number, loan consultation number, or health consultation number.

[0017] When establishing a voice call session based on a called number, the called number can be matched with the number routing configuration to determine the voice interaction node. The voice interaction node can be a voice robot node, an interactive voice response node, or a human agent voice node. After the voice interaction node returns to a ringing or connected state, a voice call session is formed between the terminal device and the voice interaction node. The call session identifier can be generated by combining terminal device information, the called number, an access timestamp, and the voice interaction node identifier, and is bound to the voice call session.

[0018] A centralized session control approach can be adopted. After a terminal device dials the fintech customer service number, the voice call request enters the session control server. The session control server parses the terminal device information and the called number, and queries the number routing configuration based on the called number. When the called number corresponds to credit card services, the voice call request is assigned to the credit card voice interaction node; when the called number corresponds to loan inquiries, the voice call request is assigned to the loan voice interaction node.

[0019] Alternatively, a separate approach can be adopted, separating the access gateway and the session control node. After receiving a voice call request, the access gateway parses the terminal device information and the called number, and writes the parsing result into the call access signaling. Once the call access signaling reaches the session control node, the session control node selects a voice interaction node based on the called number. The call session identifier can be pre-generated by the access gateway or generated by the session control node when the voice call session is established.

[0020] In the healthcare sector, when a user dials a health consultation service number, and the called number corresponds to a health follow-up service, the voice call request is assigned to a health follow-up voice interaction node, and a voice call session is established. After the call session identifier is bound, the patient's name, medication delivery address, or report query password can belong to the same voice call.

[0021] In the fintech business, when a customer dials an account service number or a loan service number, if the called number corresponds to account services, the voice call request is assigned to the account voice interaction node; if the called number corresponds to loan services, the voice call request is assigned to the loan voice interaction node. After the call session identifier is bound, the account name, account opening address, payee information, or policy service information can be associated with the same voice call.

[0022] This embodiment extracts terminal device information and called number from voice call requests and establishes a voice call session based on the called number, which can make a call have a clear terminal source and service entry point; by generating and binding a call session identifier, subsequent interactive data in the same voice call can have a unified attribution basis, thereby reducing data mismatch caused by unclear call attribution.

[0023] S20, after the voice call session is established, text interaction permission is verified based on the call session identifier and the terminal device information, and a text interaction session bound to the voice call session is established when the text interaction permission verification is passed. In this embodiment, after a voice call session is established, the call session identifier can be obtained from the session state record, call access signaling, or the session context of the session control node. The terminal device information can come from the voice call request parsing result, or from the terminal identifier, terminal address, calling number, and terminal capability marker stored in the session state record. The call session identifier is used to distinguish the current call, and the terminal device information is used to determine whether the current terminal has text interaction capabilities. Both participate in the text interaction permission verification, which can avoid the mixing of permission data between different calls.

[0024] Text interaction permission verification can be completed by reading service registration data, terminal capability data, and session status data. Service registration data may include text interaction activation flags, service entry flags, and service availability status; terminal capability data may include data channel capabilities, interface display capabilities, and input control support capabilities; session status data may include whether the voice call session is connected, maintained, and in a state where interaction can continue. After matching the above data with the call session identifier and terminal device information, the text interaction permission verification result is generated.

[0025] When text interaction permission verification passes, a text interaction session identifier is generated and bound to the call session identifier. The text interaction session can record terminal device information, call session identifier, text interaction session identifier, creation timestamp, and session validity status. Through this binding relationship, voice call sessions and text interaction sessions can maintain correspondence within the same call, ensuring that subsequent interface displays and text input belong to the same call process.

[0026] A separate verification method can be used in the text interaction backend. After the voice call session is established, the session control node generates a call arrival message, which carries the call session identifier and terminal device information. Upon receiving the call arrival message, the text interaction backend reads the terminal service registration data and terminal capability data, using the text interaction activation flag, data channel capabilities, and voice call session status as verification inputs to generate a text interaction permission verification result. If the text interaction permission verification passes, the text interaction backend creates a text interaction session and saves the binding relationship between the text interaction session and the voice call session.

[0027] Alternatively, a combination of a session control node and a capability query service can be used. After a voice call session is established, the session control node sends the call session identifier and terminal device information to the capability query service. The capability query service returns information on whether the terminal supports interface display, text input, and whether text interaction is allowed at the current service entry point. The session control node triggers the creation of a text interaction session based on the returned results. For scenarios with a high volume of concurrent calls, terminal capability data can be cached in a short-term storage area with a session-level validity period to reduce redundant queries.

[0028] In the healthcare sector, after a voice call session is established, the health consultation platform reads the current call status based on the call session identifier and determines whether the terminal supports a text input interface based on the terminal device information. If the terminal supports text interaction and the current consultation entry allows input of the patient's name, medication delivery address, or report query password, a text interaction session bound to the voice call session is established, enabling subsequent text entry to be attributed to the current consultation call.

[0029] In the fintech business, after a customer dials an account service or loan consultation number, a voice call session is established. The service platform confirms the current call status based on the call session identifier and checks whether the terminal has data interaction and interface display capabilities based on the terminal device information. If the text interaction permission verification passes, a text interaction session is established to accept subsequent text input of account name, account opening address, payee information, or policy service information.

[0030] This embodiment uses both call session identifier and terminal device information to participate in text interaction permission verification. It can determine whether the current call has the conditions for text interaction after the voice call session is established. When the verification is successful, a text interaction session bound to the voice call session is established. This can keep the voice call and text interaction under the same session ownership, thereby reducing data mismatch caused by unauthorized terminals entering the text interaction process and incorrect session ownership.

[0031] S30, based on the text interaction session, send an interface establishment request to the terminal device, so that the terminal device can display a visual interactive interface while maintaining the voice call session; In this embodiment, the text interaction session is used to handle visual data interaction within a voice call session. The text interaction session can store a call session identifier, terminal device information, session validity status, and interface resource index. When sending an interface creation request to the terminal device based on the text interaction session, the display capability flag, data channel capability flag, and terminal type flag in the terminal device information can be read first, and then interface resources can be selected accordingly. The interface creation request can carry a text interaction session identifier, a call session identifier, an interface template identifier, display area parameters, input area parameters, and a call hold flag. After receiving the interface creation request, the terminal device loads interface resources according to the interface template identifier and maintains the voice call session in an connected state according to the call hold flag. The visual interactive interface can include a session text display area, a service prompt area, and a text input area. The session text display area is used to display the textualized result of the voice interaction content, the service prompt area is used to display the current interaction prompt, and the text input area is used to accept subsequent precise text input. The key to this process is that the interface creation request is triggered by the text interaction session, and the voice call session is continuously maintained during the interface display, enabling the terminal device to load the visual interactive interface without interrupting the call.

[0032] In the healthcare sector, once a user dials a health consultation service number, a voice call session is established. The server sends an interface creation request to the terminal device based on the text-based interaction session. The terminal device then displays a visual interactive interface while maintaining the voice call session. This interface can display health consultation prompts, a patient's name input area, and a medication delivery address input area, providing independent input locations for text content that requires precise input during the voice interaction.

[0033] In the fintech business, once a customer dials an account service number or loan consultation number, a voice call session is established. The server sends an interface creation request to the terminal device based on the text-based interactive session. The terminal device then displays a visual interactive interface while maintaining the voice call session. This interface can display account information confirmation prompts, an account opening link address confirmation area, a payee information input area, or a policy service information input area, allowing text content that is difficult to consistently read during a voice call to be handled through a visual interface.

[0034] This embodiment generates and sends an interface establishment request through a text interaction session, enabling the terminal device to display a visual interactive interface while the voice call session remains connected. Since the interface establishment request carries interface resources and call hold-up information, the terminal device can load the session text display area, service prompt area, and text input area without interrupting the voice interaction, thereby providing an operable interface environment for subsequent precise text information input.

[0035] S40, Generate speech recognition text based on the voice call session, and determine the current business node and text input task based on the speech recognition text; In this embodiment, voice data is continuously generated during a voice call session. This voice data can be segmented into voice segments according to the acquisition time, and each segment is marked with a call session identifier and a segment timing marker. The call session identifier distinguishes different calls, and the segment timing marker restores the sequential relationship between the voice segments. The voice segments can be sent to a speech recognition service, which may include an acoustic feature extraction unit, a speech-to-text transcription unit, and a text concatenation unit. The acoustic feature extraction unit extracts spectral features, phoneme features, or voiceprint-independent features. The speech-to-text transcription unit converts the voice segments into text segments, and the text concatenation unit forms the speech recognition text according to the segment timing markers. The speech recognition text may include a call session identifier, a recognition time marker, and a text confidence marker to facilitate subsequent judgment on whether the voice content meets the current business processing requirements. The current business node can be determined based on the business intent, keywords, interaction rounds, and session progress status in the speech recognition text. The natural language processing service may include a word segmentation unit, an entity extraction unit, an intent recognition unit, and a node matching unit. The word segmentation unit segments the speech recognition text into terms; the entity extraction unit extracts candidate information such as name, address, account name, drug name, department name, and payee name; the intent recognition unit identifies interactive intents such as query, confirmation, processing, modification, and submission; and the node matching unit matches the interactive intent and session progress status with the business node mapping table to obtain the current business node. Text input tasks can be generated when the speech recognition text lacks the information required for the current business node, or when the speech recognition text contains low-confidence proper nouns. A text input task can include a call session identifier, the current business node, the type of information to be collected, a task identifier, and input prompts. The type of information to be collected indicates the category of content that needs to be supplemented via text, and the task identifier distinguishes different text input tasks within the same call.

[0036] This embodiment converts voice data in a voice call session into speech-recognized text with call attribution, and determines the current business node and text input task based on the speech-recognized text. This allows for timely identification of business locations requiring supplementary text input when the voice content cannot reliably carry accurate text information. Since the text input task corresponds to the current business node and call session identifier, subsequent text input can be based on the current call and current business processing requirements, reducing information collection errors caused by speech-recognition text deviation.

[0037] S50, based on the text input task, receive the text information to be verified through the visual interactive interface, generate a digital reference identifier bound to the text information to be verified, write the digital reference identifier, the text information to be verified and the call session identifier into the text mapping cache to generate a text mapping record, generate a digital input message based on the digital reference identifier, and send the digital input message to the session control node; In this embodiment, the text input task can carry a task identifier, input object type, input prompt content, and call session identifier, used to form a text input entry matching the current voice call in the visual interactive interface. After receiving the text input task, the visual interactive interface can load the text input box, submit control, and task feedback field, ensuring that the content submitted by the terminal device corresponds to the text input task. The text information to be verified represents the text content entered through the visual interactive interface, which can be a name, address, organization name, account name, payee name, drug name, or delivery address, etc., requiring precise input. When receiving the text information to be verified, the submission timestamp, task identifier, and interface session identifier can be collected simultaneously to determine whether the text content comes from the current text input task.

[0038] A numeric reference identifier is used to replace the text information to be verified transmitted on the telephone input side. It can be generated by a task identifier, a call session identifier, a submission timestamp, and a random sequence. After the numeric reference identifier is bound to the text information to be verified, the text information does not need to directly enter the telephone keypad input channel; it can use the numeric reference identifier as the retrieval entry point. The text mapping cache is used to store the corresponding data between the numeric reference identifier, the text information to be verified, and the call session identifier. It can be implemented using a memory cache, a distributed cache, or a session-level cache table. The text mapping record can contain the numeric reference identifier, the text information to be verified, the call session identifier, the task identifier, the write timestamp, and the validity status, enabling subsequent readings to locate the text content by call attribution and reference identifier.

[0039] Numeric input messages can be derived from numeric reference identifiers, and their content can be in telephone keypad input format, session event format, or call control message format. When a numeric input message is sent to the session control node, it can carry both a numeric reference identifier and a call session identifier, or it can carry only a numeric reference identifier, with the session control node supplementing the call session identifier based on the session context. After receiving a numeric input message, the session control node can continue to recognize the numeric input content while maintaining the voice call control flow, providing a trigger for subsequent retrieval of text mapping records from the text mapping buffer.

[0040] This embodiment binds the text information to be verified with a digital reference identifier, and writes the digital reference identifier, the text information to be verified, and the call session identifier into a text mapping cache. This can convert text content into a digital reference relationship that can be transmitted by the telephone input side. By generating a digital input message based on the digital reference identifier and sending it to the session control node, the session control flow can receive text input events without directly carrying long text or Chinese characters, thereby reducing the risk of distortion in the transmission of accurate text information and incorrect session attribution during voice calls.

[0041] S60, after the digital input message arrives at the session control node, the digital reference identifier is obtained based on the digital input message, the text mapping record is read from the text mapping cache according to the call session identifier and the digital reference identifier, the text information to be verified is extracted from the text mapping record, and an information verification result is generated according to the current service node, the speech recognition text and the text information to be verified.

[0042] In this embodiment, after a numeric input message arrives at the session control node, the numeric payload field can be parsed from the message body, key event field, or session event field. The numeric payload field carries a numeric reference identifier, which points to a text mapping record in the text mapping cache. During parsing, the message source, message format, and associated call status can be verified first, and then the numeric reference identifier can be obtained from the numeric payload field to prevent invalid key data from being used to read text content.

[0043] The call session identifier can be read from the session context saved by the session control node, or it can be passed along with the numeric input message. The call session identifier and the numeric reference identifier together form the cache read condition, used to locate the text mapping record in the text mapping cache. The text mapping cache can be a memory cache, a distributed cache, or a session cache table. The text mapping record can store the numeric reference identifier, the text information to be verified, the call session identifier, the business node marker, and the call status. After reading the text mapping record, the session marker, reference marker, and call status in the record can be verified, and then the text information to be verified can be extracted.

[0044] Information verification results can be generated jointly by the current business node, the speech-recognized text, and the text information to be verified. The current business node is used to determine the type of information to be verified, the speech-recognized text is used to provide the voice interaction content, and the text information to be verified is used to provide the precise text content input through the interface. When generating information verification results, the consistency between candidate content in the speech-recognized text and the text information to be verified can be compared, and it can also be checked whether the text information to be verified meets the field completeness, character format, and business semantic requirements of the current business node.

[0045] This embodiment obtains a digital reference identifier through digital input messages and reads text mapping records from the text mapping cache in conjunction with the call session identifier. This allows the digital content input from the telephone keypad to be restored to the corresponding text information to be verified. By generating an information verification result together with the current business node, the voice-recognized text, and the text information to be verified, the voice content and the interface input content can be verified under the same business node, reducing the deviation caused by relying solely on voice recognition for accurate text information.

[0046] In one embodiment, step S10 above includes: S101, receive a voice call request sent by a terminal device, read the terminal identifier field, terminal address field and called number field from the voice call request, combine the terminal identifier field and the terminal address field into terminal device information, and extract the called number from the called number field; S102, a call session identifier is generated based on the called number and the terminal device information, and a call access signaling is generated based on the called number, the terminal device information and the call session identifier. The call access signaling is sent to the session control node so that the voice call request reaches the session control node in the form of the call access signaling. S103, after the voice call request reaches the session control node, the voice interaction node identifier is read from the number routing configuration according to the called number, and the voice interaction node is determined based on the voice interaction node identifier; S104, generate call allocation signaling based on the call session identifier, the terminal device information, the called number and the voice interaction node, and transmit the call allocation signaling to the voice interaction node; S105, obtain the ringing response status and connection status returned by the voice interaction node according to the call allocation signaling. When the ringing response status indicates that the voice interaction node has been ringing and the connection status indicates that the voice interaction node has been connected, establish a voice call session between the terminal device and the voice interaction node. S106, bind the call session identifier to the voice call session.

[0047] In this embodiment, the terminal device can be implemented as a mobile terminal, a fixed-line terminal, a call center terminal, or a service terminal with voice call capability. The voice call request is initiated by the terminal device and may carry a terminal identifier field, a terminal address field, and a called number field. The terminal identifier field is used to distinguish the source of the calling terminal, the terminal address field is used to record the terminal's access location or network address, and the called number field is used to record the service number dialed by the terminal device. After receiving the voice call request, the fields in the voice call request are parsed, the terminal identifier field and the terminal address field are combined into terminal device information, and the called number is extracted from the called number field. The terminal device information may simultaneously include the calling terminal identifier, terminal access address, terminal network type, and call source marker. The called number can be included in the subsequent call allocation process after number standardization.

[0048] A call session identifier can be generated either before or after a voice call is connected. To reduce confusion regarding subsequent session data attribution, the call session identifier can be generated based on the called number and terminal device information, and then written into the call access signaling. The call access signaling can include the called number, terminal device information, call session identifier, and call access timestamp. After the call access signaling is sent to the session control node, the voice call request is received by the session control node in the form of call access signaling, enabling the session control node to allocate subsequent calls based on the same set of call access data.

[0049] The session control node can be implemented by a call control service, a session control server, or a cloud communication session control unit. When a voice call request reaches the session control node, it reads the number routing configuration based on the called number. The number routing configuration can store mapping data between the called number and the voice interaction node identifier. The voice interaction node identifier is used to locate the specific voice robot node, interactive voice response node, or human agent voice node that accepts the call. After determining the voice interaction node based on the voice interaction node identifier, the call session identifier, terminal device information, called number, and voice interaction node can be written into the call allocation signaling, and the call allocation signaling can be transmitted to the voice interaction node.

[0050] After receiving a call allocation signaling message, the voice interaction node can return to a ringing response state and an answering state. The ringing response state indicates that the voice interaction node has received the call allocation signaling message and entered the ringing phase, while the answering state indicates that a call has been established between the terminal device and the voice interaction node, allowing for the transmission of voice data. When the ringing response state indicates that the voice interaction node has rang, and the answering state indicates that the voice interaction node has been connected, a voice call session is established between the terminal device and the voice interaction node. The voice call session can record terminal device information, called number, voice interaction node, call status, and media transmission status. After a call session identifier is bound to a voice call session, subsequent data within the same call can be identified by its call session identifier.

[0051] This embodiment parses terminal device information and called number from voice call requests and determines voice interaction nodes based on the called number, enabling the call allocation process to have a clear terminal source and number entry point. By generating a call session identifier during the call access phase and binding the call session identifier to the voice call session after the voice call session is established, the session state, voice interaction objects, and subsequent interaction data in the same call can be uniformly marked, thereby reducing the probability of mixing different call data and incorrect session attribution.

[0052] In one embodiment, step S20 above includes: S201, after the voice call session is established, a call incoming message carrying a call session identifier and terminal device information is generated, and the call incoming message is sent to the text interaction processing service. S202, the call session identifier and terminal device information are read from the call incoming message through the text interaction processing service, terminal service registration data is obtained based on the terminal device information, and the session connection status of the voice call session is obtained based on the call session identifier. S203, Read the text interaction activation flag and the call data interaction capability flag from the terminal service registration data, and combine the text interaction activation flag, the call data interaction capability flag and the session connection status into permission verification parameters; S204, perform text interaction permission verification based on the permission verification parameters, and generate text interaction permission verification result; S205, when the text interaction permission verification result indicates that the text interaction permission verification has passed, a text interaction session identifier is generated based on the call session identifier, and the text interaction session identifier is bound to the call session identifier; S206, establish a text interaction session bound to the voice call session based on the text interaction session identifier, the call session identifier, and the terminal device information.

[0053] In this embodiment, after a voice call session is established, a call arrival message can be generated by the session control node or a service component connected to the session control node. The call arrival message is used to transmit the connected call status to the text interaction processing service. The message content includes a call session identifier and terminal device information. The call session identifier is used to locate the same voice call session, and the terminal device information is used to identify the source and capabilities of the terminal initiating the call. The call arrival message can be implemented using a session event message, a service call request, or a message queue event. The message can also include a connection time marker, a called number marker, and a session status marker, enabling the text interaction processing service to determine whether the current voice call session is in a state where a text interaction session can be established.

[0054] After receiving an incoming call message, the text interaction processing service reads the call session identifier and terminal device information from the message. Terminal device information may include the terminal identifier, terminal address, caller ID, terminal type, and data interaction capability flag. The text interaction processing service can query terminal service registration data based on the terminal device information. This data records whether the terminal has enabled text interaction service, supports interface display, supports text input, and allows data interaction during a call. The text interaction processing service can also query the session connection status of a voice call based on the call session identifier. The session connection status is used to exclude calls that are not connected, have been hung up, or have abnormal status.

[0055] The text interaction enable flag and the call data interaction capability flag are read from the terminal service registration data. The text interaction enable flag indicates whether text interaction is allowed for the current terminal or the current service entry point, while the call data interaction capability flag indicates whether the terminal device can receive interface data and submit text data during a voice call. The session connection status, text interaction enable flag, and call data interaction capability flag are combined into permission verification parameters, forming a unified verification input. Permission verification parameters can be stored as key-value sets, structured messages, or memory objects, facilitating the text interaction processing service to read each parameter according to the same verification process.

[0056] Text interaction permission verification can be implemented by comparing parameters one by one. The text interaction activation flag needs to indicate that the text interaction service is enabled; the call data interaction capability flag needs to indicate that the terminal device has data interaction capabilities; and the session connection status needs to indicate that the voice call session is connected. The results of these judgments can be combined into a text interaction permission verification result. The text interaction permission verification result can include a pass status, a reason for failure, and a verification time stamp. The reason for failure can be recorded as service not enabled, terminal not supporting data interaction, or the voice call session ending, thus facilitating subsequent status recording and anomaly troubleshooting.

[0057] The text interaction permission verification result indicates that when the text interaction permission verification is successful, a text interaction session identifier can be generated based on the call session identifier. The text interaction session identifier can be generated by combining the call session identifier, creation timestamp, and a random sequence, or it can be assigned by the text interaction processing service. After the text interaction session identifier is bound to the call session identifier, the text interaction session can form a corresponding relationship with the voice call session within the same call range. The text interaction session can save the text interaction session identifier, call session identifier, terminal device information, session validity status, and creation timestamp, so that subsequent interface establishment requests can be sent according to the text interaction session.

[0058] This embodiment triggers text interaction permission verification by using call session identifier and terminal device information. It can jointly determine the terminal service registration data, call data interaction capability, and session connection status, provided that a voice call session has been established. By generating a text interaction session identifier when the text interaction permission verification is passed and binding the text interaction session identifier with the call session identifier, the text interaction session and the voice call session can maintain consistent session affiliation. This reduces the number of terminals that do not meet the conditions for text interaction from entering the text interaction process and reduces the risk of mismatch between text interaction data and voice call session.

[0059] In one embodiment, step S30 above includes: S301, obtain the interface resource configuration based on the text interaction session and the terminal device information, and read the interface template identifier, session text display area configuration, business prompt area configuration and text input area configuration from the interface resource configuration; S302, Based on the text interaction session, the call session identifier, the interface template identifier, the session text display area configuration, the service prompt area configuration, and the text input area configuration, generate an interface creation request, and write the call hold flag of the voice call session into the interface creation request; S303, establish a call data bearer channel bound to the text interaction session, and send the interface creation request to the terminal device through the call data bearer channel; S304, after the terminal device receives the interface establishment request, it maintains the voice call session based on the call hold flag and loads the interface rendering resources according to the interface template identifier; S305, the configuration of the conversation text display area, the configuration of the service prompt area, and the configuration of the text input area are written into the interface rendering resources to generate a visual interactive interface, and the terminal device displays the visual interactive interface while maintaining the voice call session.

[0060] In this embodiment, the text interaction session stores the call session identifier, terminal device information, session validity status, and interface resource index. When obtaining the interface resource configuration based on the text interaction session and terminal device information, the server can read the terminal screen type, data carrying capacity, input control capabilities, and interface version flag, and select the interface resource configuration that can be loaded on the terminal device accordingly. The interface resource configuration includes an interface template identifier, a session text display area configuration, a service prompt area configuration, and a text input area configuration. The interface template identifier points to an interface template that can be rendered by the terminal device, the session text display area configuration determines the textual display position of the voice interaction content, the service prompt area configuration determines the display position of the prompt content, and the text input area configuration determines the position, input type, and submission method of the text input controls.

[0061] The interface establishment request is generated jointly by the text interaction session, the call session identifier, and the interface resource configuration. The text interaction session is used to mark the text interaction process to which the interface establishment request belongs, and the call session identifier is used to mark the voice call process to which the interface establishment request belongs. After the call hold flag is written into the interface establishment request, the terminal device can maintain the connected state of the voice call session while loading the interface, avoiding call interruption or call switching triggered by the interface loading action. The interface establishment request can be carried by a session event message, a data channel message, or a push message. The message content includes the interface template identifier, region configuration, call hold flag, and text interaction session identifier.

[0062] After the call data bearer channel is bound to the text interaction session, it is used to transmit the interface establishment request to the terminal device. The call data bearer channel can be implemented through the operator's data channel, the in-application real-time data channel, or the cloud communication data channel. After receiving the interface establishment request, the terminal device maintains the voice call session according to the call hold flag and loads the interface rendering resources according to the interface template identifier. The interface rendering resources may include the interface layout file, control configuration file, and display style file. After the session text display area configuration, service prompt area configuration, and text input area configuration are written into the interface rendering resources, the terminal device generates a visual interactive interface and displays the visual interactive interface while maintaining the voice call session.

[0063] This embodiment obtains the interface resource configuration through a text interaction session and writes the interface template identifier, display area configuration, input area configuration, and call hold flag into the interface creation request. This allows the terminal device to load a visual interactive interface while maintaining a voice call session. By sending the interface creation request through the call data bearer channel bound to the text interaction session, the interface display process can be kept in correspondence with the current voice call process, thereby reducing input attribution errors caused by mismatch between interface display and call session, and providing a stable interface bearing environment for subsequent accurate text input.

[0064] In one embodiment, step S40 above includes: S401, extract a sequence of voice segments from the voice call session according to the acquisition time, and write the call session identifier and segment timing mark into the voice segment sequence to generate a voice recognition request; S402, the speech recognition request is sent to the speech recognition service, and the speech segment sequence is transcribed according to the segment time sequence marker to generate a set of recognized text segments; S403, the set of recognized text segments is spliced ​​together according to the segment time sequence marker to generate speech recognition text, and the speech recognition text is bound to the call session identifier; S404, the speech recognition text is sent to a natural language processing service to extract business intent information and information type to be collected from the speech recognition text; S405, obtain the session process status of the voice call session, and determine the current service node from the service node mapping table based on the service intent information and the session process status; S406, Based on the type of information to be collected and the information integrity conditions corresponding to the current business node, perform information integrity verification on the speech recognition text and generate information integrity verification results; S407, when the type of information to be collected belongs to the precise text information type, or when the information integrity verification result indicates that the speech recognition text has failed the information integrity verification, a task identifier is generated; S408, construct a text input task and write the current business node, the type of information to be collected, the call session identifier, and the task identifier into the text input task.

[0065] In this embodiment, the voice data generated during a voice call session can be segmented into a sequence of voice segments according to the acquisition time. The voice segment sequence can be extracted using a fixed time window, a silence interval, or a semantic pause. A fixed time window is suitable for continuous call streams, a silence interval is suitable for question-and-answer interactions, and a semantic pause is suitable for scenarios where the customer speaks a long message at once. After each voice segment is written with a call session identifier and a segment timing marker, the attribution relationship between the voice segment and the voice call session can be maintained, and the sequential position of the voice segment within the call can be preserved. The voice recognition request can carry the voice segment sequence, call session identifier, segment timing marker, and audio format marker, facilitating the voice recognition service to transcribe the data according to call attribution and segment order.

[0066] A speech recognition service can include an audio preprocessing unit, an acoustic feature extraction unit, a speech-to-text unit, and a text post-processing unit. The audio preprocessing unit performs noise reduction, silence trimming, and volume normalization. The acoustic feature extraction unit extracts spectral features, phoneme features, and speech rate features. The speech-to-text unit converts a sequence of speech segments into a set of recognized text segments. The text post-processing unit sorts, removes duplicates, and completes punctuation on the recognized text segments based on their temporal markers, then concatenates them to generate the speech-recognized text. After the speech-recognized text is bound to a call session identifier, the call attribution can be maintained in subsequent semantic analysis, preventing text content from different voice call sessions from being mixed in the same judgment process.

[0067] Natural Language Processing (NLP) services can include word segmentation, entity extraction, intent recognition, and node matching units. After speech recognition text enters the NLP service, the word segmentation unit divides the text into word sequences. The entity extraction unit extracts candidate entities such as names, addresses, organization names, account names, drug names, and department names from these sequences. The intent recognition unit identifies business intent information such as query, confirmation, modification, and submission based on the word sequences and candidate entities. The type of information to be collected can be determined by the candidate entity type, prompt words in the speech recognition text, and business intent information, indicating the category of information to be collected in the current voice call session.

[0068] The session progress state records the completed interaction rounds, the previous service node, the current prompt content, and the input waiting status during a voice call session. The service node mapping table stores the correspondence between service intent information, session progress state, and service nodes. The natural language processing service matches the service intent information and session progress state against the service node mapping table to determine the current service node. The current service node indicates the location of information collection that needs to be completed in the voice call session, such as name confirmation, address confirmation, organization information confirmation, or service recipient information confirmation.

[0069] Information integrity conditions can include whether fields are missing, whether field lengths meet requirements, whether candidate entity types match the current business node, and whether the confidence level of the target content in the speech-recognition text meets requirements. Based on the information integrity conditions corresponding to the type of information to be collected and the current business node, information integrity verification can be performed on the speech-recognition text, and information integrity verification results can be generated. When the type of information to be collected belongs to the exact text information type, or when the information integrity verification result shows that the speech-recognition text fails the information integrity verification, a task identifier can be generated and a text input task can be constructed. After writing the current business node, the type of information to be collected, the call session identifier, and the task identifier into the text input task, the voice call session to which the text input task belongs, the corresponding business location, and the category of text content to be collected can be clearly identified.

[0070] This embodiment segments the voice data in a voice call session into a sequence of voice segments with call session identifiers and segment timing markers. The voice recognition service then generates voice recognition text bound to the call session identifiers, maintaining the session affiliation and text order of the voice recognition results. By extracting business intent information and the type of information to be collected from the voice recognition text through a natural language processing service, and determining the current business node in conjunction with the session progress status, the location of the information to be collected in the current voice call session can be located. Information integrity verification determines whether the voice recognition text meets the requirements of the current business node, and a text input task is constructed when precise text information is required. This reduces the collection deviation caused by relying solely on voice recognition for content such as names, addresses, and organization names.

[0071] In one embodiment, step S50 above includes: S501, read the task identifier, current business node, information type to be collected and call session identifier from the text input task, and read the submission session identifier from the text interaction session, generate a text submission control based on the task identifier, the current business node and the information type to be collected, and load the text submission control into the visual interactive interface; S502, the terminal device submits text information to be verified, return task identifier and return session identifier through the text submission control in the visual interactive interface, and collects the submission time marker corresponding to the text information to be verified. S503, perform consistency verification between the return task identifier and the task identifier, perform consistency verification between the return session identifier and the submission session identifier, and perform collection item verification based on the current business node and the type of information to be collected, and generate submission verification result; S504, when the submission verification result indicates that the submission is successful, a digital reference identifier bound to the text information to be verified is generated based on the task identifier, the call session identifier, and the submission time stamp; S505, the digital reference identifier, the text information to be verified, the call session identifier, the task identifier, the current business node, and the information type to be collected are written into the text mapping cache area to generate a text mapping record; S506, Generate a numeric input message conforming to the telephone keypad input format based on the numeric reference identifier, associate the numeric input message with the call session identifier, and send the numeric input message to the session control node through the telephone input interface.

[0072] In this embodiment, the text input task can store a task identifier, the current business node, the type of information to be collected, and a call session identifier. The task identifier is used to distinguish different text input requests within the same voice call session; the current business node is used to indicate the interaction position corresponding to the text content; the type of information to be collected is used to indicate the category of content to be input; and the call session identifier is used to maintain the relationship between the text input task and the voice call session. After reading the submission session identifier from the text interaction session, the submission session identifier can be used as a submission marker for the visual interactive interface, enabling the terminal device to return the corresponding session marker when submitting text content. When generating a text submission control based on the task identifier, the current business node, and the type of information to be collected, the input box type, prompt content, field validation method, and submit button state can be determined. Then, the text submission control is loaded onto the visual interactive interface, allowing the terminal device to input text information to be validated in the current interface.

[0073] When the text submission control in the visual interactive interface receives data submitted by the terminal device, it can simultaneously receive the text information to be verified, the return task identifier, and the return session identifier. The text information to be verified is the text content entered by the terminal device, which may include name, address, organization name, account name, payee name, drug name, or delivery address. The return task identifier is used for consistency verification with the task identifier in the text input task, and the return session identifier is used for consistency verification with the submission session identifier in the text interaction session. The submission timestamp can be generated when the server receives the text information to be verified, or it can be uploaded synchronously by the terminal device when submitting. The submission timestamp, task identifier, and call session identifier jointly participate in the generation of the digital reference identifier, enabling the digital reference identifier to correspond to a specific text submission action.

[0074] Consistency verification is used to determine whether the text information to be verified comes from the current text input task and the current text interaction session. When the returned task identifier matches the current task identifier, it indicates that the data submitted by the terminal device corresponds to the current text input task; when the returned session identifier matches the submitted session identifier, it indicates that the submitted data comes from the current text interaction session. Item verification can be performed based on the current business node and the type of information to be collected. For example, it can determine whether the current business node needs to collect this type of text content, whether the text information to be verified contains null values, and whether the field length and character type meet the input requirements. The above verification results are combined to form the submission verification result. When the submission verification result indicates that the submission has passed, a digital reference identifier is generated based on the task identifier, call session identifier, and submission timestamp, and this digital reference identifier is bound to the text information to be verified. The digital reference identifier can be generated using an incrementing number, a random number string, or a unique number string within the session. The generated result must meet the requirements for carrying subsequent digital input messages.

[0075] The text mapping cache stores the mapping relationship between numeric reference identifiers and text information to be verified. After writing the numeric reference identifier, the text information to be verified, the call session identifier, the task identifier, the current business node, and the type of information to be collected into the text mapping cache, a text mapping record is formed. The text mapping record can include a write timestamp, validity status, and call status to determine if the record is still usable during subsequent readings. Numeric input messages are generated by numeric reference identifiers and can use a telephone keypad input format, enabling the session control node to receive numeric content according to telephone input events. After the numeric input message is associated with the call session identifier, it is sent to the session control node through the telephone input interface, so that the text content does not need to directly enter the telephone input channel; the telephone input channel only carries the numeric content corresponding to the numeric reference identifier.

[0076] This embodiment generates a text submission control through a text input task and returns a task identifier and a session identifier upon submission, ensuring that the text information to be verified corresponds to the current text input task and the current text interaction session. After consistency verification and collection item verification, a digital reference identifier is generated, and the digital reference identifier, the text information to be verified, and the call session identifier are written into the text mapping cache, which can convert the text content into a searchable digital reference relationship. The digital reference identifier is sent to the session control node through a digital input message conforming to the telephone keypad input format, which allows the session control node to receive digital content in the telephone input channel while retaining the complete content of the text information to be verified in the text mapping cache, thereby reducing recognition errors and content loss caused by accurately transmitting text information directly through voice or telephone keypad.

[0077] In one embodiment, step S60 above includes: S601, after the digital input message arrives at the session control node, the digital payload field is extracted from the digital input message, and the digital reference identifier is obtained based on the digital payload field; S602, read the call session identifier corresponding to the numeric input message from the session context of the session control node, and generate a cache read index based on the call session identifier and the numeric reference identifier; S603, read the text mapping record from the text mapping cache area according to the cache read index, and read the record session identifier, record reference identifier, record business node, record call status and cached text information from the text mapping record; S604, perform consistency verification between the record session identifier and the call session identifier to generate a session consistency verification result; perform consistency verification between the record reference identifier and the digital reference identifier to generate a reference consistency verification result; perform consistency verification between the record service node and the current service node to generate a service node consistency verification result; and generate a record verification result based on the session consistency verification result, the reference consistency verification result, the service node consistency verification result, and the record call status. S605, when the record verification result characterization passes, extract the cached text information from the text mapping record as the text information to be verified, and update the record call status in the text mapping record to the called status; S606, based on the information verification conditions read by the current business node, perform text consistency verification and field integrity verification on the speech recognition text and the text information to be verified according to the information verification conditions, and generate information verification results.

[0078] In this embodiment, after a numeric input message arrives at the session control node, the message body can be read from a telephone input event, key input message, or session event message, and the numeric payload field can be extracted from the message body. The numeric payload field carries a numeric reference identifier and can be a numeric string, key sequence, or encoded session input value. Before extracting the numeric payload field, the message source, message timestamp, and session reception status can be verified, and input data not belonging to the current session control node can be filtered out. After the numeric payload field is extracted, it is restored according to the encoding format of the numeric input message to obtain the numeric reference identifier, enabling the numeric content in the numeric input message to be matched with records in the text mapping buffer.

[0079] The call session identifier can be read from the session context saved by the session control node or carried along with the numeric input message. The session context can save the call session identifier, terminal device information, voice interaction node identifier, and current call status. When generating a cache read index based on the call session identifier and numeric reference identifier, the call session identifier can be used as the session scope condition, and the numeric reference identifier as the record location condition. The cache read index can use a composite key, structured query conditions, or cache table primary key format, enabling the text mapping cache to read the corresponding text mapping record within the same voice call session, reducing the error in record retrieval between different voice call sessions.

[0080] After a text mapping record is read from the text mapping cache, the record session identifier, record reference identifier, record business node, record call status, and cached text information can be read separately. The record session identifier is used for consistency verification with the call session identifier; the record reference identifier is used for consistency verification with the numeric reference identifier; the record business node is used for consistency verification with the current business node; and the record call status is used to determine whether the text mapping record has already been read and used. After the session consistency verification result, reference consistency verification result, and business node consistency verification result are formed, they are combined with the record call status to generate the record verification result. When the record verification result indicates that the record has passed, the cached text information is extracted as the text information to be verified, and the record call status is updated to the called state, ensuring that the same text mapping record does not participate in subsequent verifications repeatedly.

[0081] Information verification conditions can be read based on the current business node. These conditions may include target field category, field length condition, character type condition, required field status, semantic matching condition, and text consistency condition. Speech recognition text provides candidate content identified during voice calls, while the text information to be verified provides the text content submitted through the visual interactive interface. Text consistency verification compares the semantic correspondence between candidate entities in the speech recognition text and the text information to be verified. Field integrity verification determines whether the text information to be verified meets the information verification conditions corresponding to the current business node. The results of text consistency verification and field integrity verification are then combined to generate the information verification result.

[0082] This embodiment obtains the digital reference identifier from the digital input message and reads the text mapping record in conjunction with the call session identifier. This allows for the accurate reconstruction of the digital content input from the telephone input side into the corresponding text information to be verified. By recording the session identifier, the reference identifier, the business node, and the call status for verification, mismatches between the text mapping record and the current voice call session and the current business node can be reduced. By using both the voice-recognized text and the text information to be verified for text consistency verification and field integrity verification, the accurate fields can be supplemented using the text input content while retaining the voice interaction content, thereby reducing the errors in accurate text information caused by relying solely on voice recognition.

[0083] In one embodiment, a system architecture is provided for a method of collaborative voice and text interaction during a call, which can be as follows: Figure 6 As shown in the diagram, the mobile phone on the left represents the terminal device, which is used to initiate a voice call request and display a visual interactive interface during the voice call. The path marked S100 indicates that the terminal device initiates a voice call request to the access side, which carries call access data such as terminal device information and the called number.

[0084] The Gateway (GW) is located between the terminal device and the session control side, and is used to receive voice call requests and forward call access data. The path marked S200 indicates that the gateway sends the call access signaling corresponding to the voice call request to the SIP (Session Initiation Protocol) service side. The SIP service side corresponds to the session control node in the diagram, and is used to establish a voice call session between the terminal device and the voice interaction node based on the called number, and generate a call session identifier.

[0085] The ASR (Automatic Speech Recognition) on the right side of the diagram is used to receive voice data from voice call sessions and generate speech-recognized text. The path marked S300 indicates that the session control node transmits voice data to the ASR service, and the ASR service outputs speech-recognized text, providing a text source for the current business node's recognition and text input tasks.

[0086] The TTS (Text to Speech) section on the right side of the diagram is used to convert text prompts into speech prompts. The path marked S301 represents the voice broadcast data interaction between the session control node and the TTS service. This part can be used to play service prompts, text input prompts, or verification prompts during voice call sessions.

[0087] The NLP (Natural Language Processing) on ​​the right side of the diagram is used to identify the business intent, the type of information to be collected, and the current business node based on the speech recognition text. The path marked S302 indicates that the session control node transmits speech recognition text or session text data to the NLP service. The NLP service performs semantic analysis on the text content and forms a judgment result related to the current business node and text input task.

[0088] The text interaction backend in the lower left corner of the diagram is used to establish text interaction sessions, send interface creation requests to the terminal device, receive text information to be verified submitted by the visual interactive interface, and generate a numerical reference identifier. The path marked S500 indicates the text interaction data transmission between the terminal device and the text interaction backend, where the terminal device submits text information to be verified through the visual interactive interface.

[0089] Redis (Remote Dictionary Server), located at the bottom of the diagram, serves as a text mapping cache to store text mapping records. The path marked S401 indicates that the text interaction backend writes the numeric reference identifier, the text information to be verified, and the call session identifier to Redis, forming a text mapping record.

[0090] The path marked S303 indicates that the text interaction backend sends a numeric input message to the session control node. The numeric input message is generated by a numeric reference identifier, which is used by the session control node to obtain the numeric reference identifier and, in conjunction with the call session identifier, read the text mapping record from the text mapping buffer.

[0091] The path marked S400 represents the data interaction between the cache side and subsequent semantic verification processing. After obtaining the digital reference identifier based on the digital input message, the session control node can read the text mapping record from Redis according to the call session identifier and the digital reference identifier, extract the text information to be verified, and use the text information to be verified, the speech recognition text, and the current business node together to generate the information verification result.

[0092] In one embodiment, a voice and text collaborative interaction device for in-call communication is provided, which corresponds one-to-one with the voice and text collaborative interaction method for in-call communication described in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the voice and text collaborative interaction device during a call according to the present invention. The modules include a call session establishment module 10, a text permission verification module 20, a visual interface push module 30, a voice semantic recognition module 40, a text mapping and forwarding module 50, and a text information verification module 60. Detailed descriptions of each functional module are as follows: The call session establishment module 10 is used to receive a voice call request sent by a terminal device, make the voice call request reach the session control node, extract terminal device information and called number from the voice call request, establish a voice call session between the terminal device and the voice interaction node based on the called number, and generate a call session identifier corresponding to the voice call session. The text permission verification module 20 is used to perform text interaction permission verification based on the call session identifier and the terminal device information after the voice call session is established, and to establish a text interaction session bound to the voice call session when the text interaction permission verification is passed. The visual interface push module 30 is used to send an interface creation request to the terminal device based on the text interaction session, so that the terminal device can display a visual interactive interface while maintaining the voice call session. The speech semantic recognition module 40 is used to generate speech recognition text based on the voice call session, and determine the current business node and text input task based on the speech recognition text; The text mapping forwarding module 50 is used to receive text information to be verified through the visual interactive interface based on the text input task, generate a digital reference identifier bound to the text information to be verified, write the digital reference identifier, the text information to be verified and the call session identifier into the text mapping cache to generate a text mapping record, generate a digital input message based on the digital reference identifier, and send the digital input message to the session control node. The text information verification module 60 is used to obtain the digital reference identifier based on the digital input message after the digital input message arrives at the session control node, read the text mapping record from the text mapping cache according to the call session identifier and the digital reference identifier, extract the text information to be verified from the text mapping record, and generate an information verification result based on the current service node, the speech recognition text and the text information to be verified.

[0093] In one embodiment, the call session establishment module 10 is specifically used for: The system receives a voice call request sent by a terminal device, reads a terminal identifier field, a terminal address field, and a called number field from the voice call request, combines the terminal identifier field and the terminal address field into terminal device information, and extracts the called number from the called number field. A call session identifier is generated based on the called number and the terminal device information, and a call access signaling is generated based on the called number, the terminal device information and the call session identifier. The call access signaling is sent to the session control node so that the voice call request reaches the session control node in the form of the call access signaling. After the voice call request reaches the session control node, the voice interaction node identifier is read from the number routing configuration based on the called number, and the voice interaction node is determined based on the voice interaction node identifier. Based on the call session identifier, the terminal device information, the called number, and the voice interaction node, a call allocation signaling is generated, and the call allocation signaling is transmitted to the voice interaction node; The system obtains the ringing response status and connection status returned by the voice interaction node according to the call allocation signaling. When the ringing response status indicates that the voice interaction node has been rang and the connection status indicates that the voice interaction node has been connected, a voice call session is established between the terminal device and the voice interaction node. Bind the call session identifier to the voice call session.

[0094] In one embodiment, the text permission verification module 20 is specifically used for: After the voice call session is established, a call incoming message carrying a call session identifier and terminal device information is generated, and the call incoming message is sent to the text interaction processing service. The text interaction processing service reads the call session identifier and terminal device information from the call incoming message, obtains terminal service registration data based on the terminal device information, and obtains the session connection status of the voice call session based on the call session identifier. Read the text interaction activation flag and the call data interaction capability flag from the terminal service registration data, and combine the text interaction activation flag, the call data interaction capability flag and the session connection status into permission verification parameters; Based on the permission verification parameters, perform text interaction permission verification and generate text interaction permission verification results. When the text interaction permission verification result indicates that the text interaction permission verification has passed, a text interaction session identifier is generated based on the call session identifier, and the text interaction session identifier is bound to the call session identifier; A text interaction session is established based on the text interaction session identifier, the call session identifier, and the terminal device information, and is bound to the voice call session.

[0095] In one embodiment, the visual interface push module 30 is specifically used for: Based on the text interaction session and the terminal device information, obtain the interface resource configuration, and read the interface template identifier, session text display area configuration, business prompt area configuration and text input area configuration from the interface resource configuration; An interface creation request is generated based on the text interaction session, the call session identifier, the interface template identifier, the session text display area configuration, the business prompt area configuration, and the text input area configuration, and the call hold flag of the voice call session is written into the interface creation request; Establish a call data bearer channel bound to the text interaction session, and send the interface creation request to the terminal device through the call data bearer channel; After the terminal device receives the interface establishment request, it maintains the voice call session based on the call hold flag and loads the interface rendering resources according to the interface template identifier. The configuration of the conversation text display area, the configuration of the service prompt area, and the configuration of the text input area are written into the interface rendering resources to generate a visual interactive interface, and the terminal device displays the visual interactive interface while maintaining the voice call session.

[0096] In one embodiment, the speech semantic recognition module 40 is specifically used for: Extract a sequence of audio segments from the voice call session according to the acquisition time, and write the call session identifier and segment timing mark into the audio segment sequence to generate a voice recognition request; The speech recognition request is sent to the speech recognition service, and the speech segment sequence is transcribed according to the segment time sequence marker to generate a set of recognized text segments; The set of recognized text segments is spliced ​​together according to the segment time sequence markers to generate speech recognition text, and the speech recognition text is bound to the call session identifier; The speech-recognized text is sent to a natural language processing service to extract business intent information and the type of information to be collected from the speech-recognized text. Obtain the session process status of the voice call session, and determine the current service node from the service node mapping table based on the service intent information and the session process status; Based on the type of information to be collected and the information integrity conditions corresponding to the current business node, the speech recognition text is verified for information integrity, and an information integrity verification result is generated. When the type of information to be collected is a precise text information type, or when the information integrity verification result indicates that the speech recognition text has failed the information integrity verification, a task identifier is generated; Construct a text input task and write the current business node, the type of information to be collected, the call session identifier, and the task identifier into the text input task.

[0097] In one embodiment, the text mapping forwarding module 50 is specifically used for: Read the task identifier, current business node, information type to be collected, and call session identifier from the text input task, and read the submission session identifier from the text interaction session. Generate a text submission control based on the task identifier, the current business node, and the information type to be collected, and load the text submission control into the visual interactive interface. The system receives the text information to be verified, the return task identifier, and the return session identifier submitted by the terminal device through the text submission control in the visual interactive interface, and collects the submission time marker corresponding to the text information to be verified. The consistency of the return task identifier and the task identifier is verified, the consistency of the return session identifier and the submission session identifier is verified, and the collection item is verified according to the current business node and the type of information to be collected, and a submission verification result is generated. When the submission verification result is deemed successful, a digital reference identifier bound to the text information to be verified is generated based on the task identifier, the call session identifier, and the submission timestamp. Write the digital reference identifier, the text information to be verified, the call session identifier, the task identifier, the current business node, and the information type to be collected into the text mapping cache to generate a text mapping record; A numeric input message conforming to the telephone keypad input format is generated based on the numeric reference identifier. The numeric input message is associated with the call session identifier and sent to the session control node through the telephone input interface.

[0098] In one embodiment, the text information verification module 60 is specifically used for: After the digital input message arrives at the session control node, the digital payload field is extracted from the digital input message, and the digital reference identifier is obtained based on the digital payload field; Read the call session identifier corresponding to the numeric input message from the session context of the session control node, and generate a cache read index based on the call session identifier and the numeric reference identifier; Based on the cache read index, read the text mapping record from the text mapping cache area, and read the record session identifier, record reference identifier, record business node, record call status and cached text information from the text mapping record; The consistency verification results are generated by performing consistency verification between the record session identifier and the call session identifier, and by performing consistency verification between the record reference identifier and the digital reference identifier, and by performing consistency verification between the record service node and the current service node, and by performing consistency verification between the record service node and the current service node. Finally, a record verification result is generated based on the session consistency verification result, the reference consistency verification result, the service node consistency verification result, and the record call status. When the record verification result is passed, the cached text information is extracted from the text mapping record as the text information to be verified, and the record call status in the text mapping record is updated to the called status. Based on the current business node read information verification conditions, the speech recognition text and the text information to be verified are subjected to text consistency verification and field integrity verification according to the information verification conditions, and information verification results are generated.

[0099] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a voice and text collaborative interaction method on the server side during a call.

[0100] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the functions or steps of a client-side method for collaborative voice and text interaction during a call.

[0101] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: The system receives a voice call request sent by a terminal device, causes the voice call request to reach the session control node, extracts terminal device information and called number from the voice call request, establishes a voice call session between the terminal device and the voice interaction node based on the called number, and generates a call session identifier corresponding to the voice call session. After the voice call session is established, text interaction permission is verified based on the call session identifier and the terminal device information. When the text interaction permission verification is passed, a text interaction session bound to the voice call session is established. Based on the text interaction session, an interface creation request is sent to the terminal device, so that the terminal device can display a visual interactive interface while maintaining the voice call session. Based on the voice call session, speech recognition text is generated, and the current business node and text input task are determined according to the speech recognition text; Based on the text input task, the system receives text information to be verified through the visual interactive interface, generates a digital reference identifier bound to the text information to be verified, writes the digital reference identifier, the text information to be verified, and the call session identifier into the text mapping cache to generate a text mapping record, generates a digital input message based on the digital reference identifier, and sends the digital input message to the session control node. After the numeric input message arrives at the session control node, the numeric reference identifier is obtained based on the numeric input message. The text mapping record is read from the text mapping cache according to the call session identifier and the numeric reference identifier. The text information to be verified is extracted from the text mapping record. An information verification result is generated based on the current service node, the speech recognition text, and the text information to be verified.

[0102] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, performs the following steps: The system receives a voice call request sent by a terminal device, causes the voice call request to reach the session control node, extracts terminal device information and called number from the voice call request, establishes a voice call session between the terminal device and the voice interaction node based on the called number, and generates a call session identifier corresponding to the voice call session. After the voice call session is established, text interaction permission is verified based on the call session identifier and the terminal device information. When the text interaction permission verification is passed, a text interaction session bound to the voice call session is established. Based on the text interaction session, an interface creation request is sent to the terminal device, so that the terminal device can display a visual interactive interface while maintaining the voice call session. Based on the voice call session, speech recognition text is generated, and the current business node and text input task are determined according to the speech recognition text; Based on the text input task, the system receives text information to be verified through the visual interactive interface, generates a digital reference identifier bound to the text information to be verified, writes the digital reference identifier, the text information to be verified, and the call session identifier into the text mapping cache to generate a text mapping record, generates a digital input message based on the digital reference identifier, and sends the digital input message to the session control node. After the numeric input message arrives at the session control node, the numeric reference identifier is obtained based on the numeric input message. The text mapping record is read from the text mapping cache according to the call session identifier and the numeric reference identifier. The text information to be verified is extracted from the text mapping record. An information verification result is generated based on the current service node, the speech recognition text, and the text information to be verified.

[0103] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0104] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0105] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0106] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0107] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for collaborative voice and text interaction during a phone call, characterized in that, Includes the following steps: The system receives a voice call request sent by a terminal device, causes the voice call request to reach the session control node, extracts terminal device information and called number from the voice call request, establishes a voice call session between the terminal device and the voice interaction node based on the called number, and generates a call session identifier corresponding to the voice call session. After the voice call session is established, text interaction permission is verified based on the call session identifier and the terminal device information. When the text interaction permission verification is passed, a text interaction session bound to the voice call session is established. Based on the text interaction session, an interface creation request is sent to the terminal device, so that the terminal device can display a visual interactive interface while maintaining the voice call session. Based on the voice call session, speech recognition text is generated, and the current business node and text input task are determined according to the speech recognition text; Based on the text input task, the system receives text information to be verified through the visual interactive interface, generates a digital reference identifier bound to the text information to be verified, writes the digital reference identifier, the text information to be verified, and the call session identifier into the text mapping cache to generate a text mapping record, generates a digital input message based on the digital reference identifier, and sends the digital input message to the session control node. After the numeric input message arrives at the session control node, the numeric reference identifier is obtained based on the numeric input message. The text mapping record is read from the text mapping cache according to the call session identifier and the numeric reference identifier. The text information to be verified is extracted from the text mapping record. An information verification result is generated based on the current service node, the speech recognition text, and the text information to be verified.

2. The method for collaborative voice and text interaction during a call as described in claim 1, characterized in that, The system receives a voice call request from a terminal device, brings the voice call request to the session control node, extracts terminal device information and the called number from the voice call request, establishes a voice call session between the terminal device and the voice interaction node based on the called number, and generates a call session identifier corresponding to the voice call session, including: The system receives a voice call request sent by a terminal device, reads a terminal identifier field, a terminal address field, and a called number field from the voice call request, combines the terminal identifier field and the terminal address field into terminal device information, and extracts the called number from the called number field. A call session identifier is generated based on the called number and the terminal device information, and a call access signaling is generated based on the called number, the terminal device information and the call session identifier. The call access signaling is sent to the session control node so that the voice call request reaches the session control node in the form of the call access signaling. After the voice call request reaches the session control node, the voice interaction node identifier is read from the number routing configuration based on the called number, and the voice interaction node is determined based on the voice interaction node identifier. Based on the call session identifier, the terminal device information, the called number, and the voice interaction node, a call allocation signaling is generated, and the call allocation signaling is transmitted to the voice interaction node; The system obtains the ringing response status and connection status returned by the voice interaction node according to the call allocation signaling. When the ringing response status indicates that the voice interaction node has been rang and the connection status indicates that the voice interaction node has been connected, a voice call session is established between the terminal device and the voice interaction node. Bind the call session identifier to the voice call session.

3. The method for collaborative voice and text interaction during a call as described in claim 1, characterized in that, After the voice call session is established, text interaction permission verification is performed based on the call session identifier and the terminal device information. If the text interaction permission verification passes, a text interaction session bound to the voice call session is established, including: After the voice call session is established, a call incoming message carrying a call session identifier and terminal device information is generated, and the call incoming message is sent to the text interaction processing service. The text interaction processing service reads the call session identifier and terminal device information from the call incoming message, obtains terminal service registration data based on the terminal device information, and obtains the session connection status of the voice call session based on the call session identifier. Read the text interaction activation flag and the call data interaction capability flag from the terminal service registration data, and combine the text interaction activation flag, the call data interaction capability flag and the session connection status into permission verification parameters; Based on the permission verification parameters, perform text interaction permission verification and generate text interaction permission verification results. When the text interaction permission verification result indicates that the text interaction permission verification has passed, a text interaction session identifier is generated based on the call session identifier, and the text interaction session identifier is bound to the call session identifier; A text interaction session is established based on the text interaction session identifier, the call session identifier, and the terminal device information, and is bound to the voice call session.

4. The method for collaborative voice and text interaction during a call as described in claim 1, characterized in that, Based on the text interaction session, a request to establish an interface is sent to the terminal device, enabling the terminal device to display a visual interactive interface while maintaining the voice call session, including: Based on the text interaction session and the terminal device information, obtain the interface resource configuration, and read the interface template identifier, session text display area configuration, business prompt area configuration and text input area configuration from the interface resource configuration; An interface creation request is generated based on the text interaction session, the call session identifier, the interface template identifier, the session text display area configuration, the business prompt area configuration, and the text input area configuration, and the call hold flag of the voice call session is written into the interface creation request; Establish a call data bearer channel bound to the text interaction session, and send the interface creation request to the terminal device through the call data bearer channel; After the terminal device receives the interface establishment request, it maintains the voice call session based on the call hold flag and loads the interface rendering resources according to the interface template identifier. The configuration of the conversation text display area, the configuration of the service prompt area, and the configuration of the text input area are written into the interface rendering resources to generate a visual interactive interface, and the terminal device displays the visual interactive interface while maintaining the voice call session.

5. The method for collaborative voice and text interaction during a call as described in claim 1, characterized in that, Generating speech-recognized text based on the voice call session, and determining the current business node and text input task based on the speech-recognized text, including: Extract a sequence of audio segments from the voice call session according to the acquisition time, and write the call session identifier and segment timing mark into the audio segment sequence to generate a voice recognition request; The speech recognition request is sent to the speech recognition service, and the speech segment sequence is transcribed according to the segment time sequence marker to generate a set of recognized text segments; The set of recognized text segments is spliced ​​together according to the segment time sequence markers to generate speech recognition text, and the speech recognition text is bound to the call session identifier; The speech-recognized text is sent to a natural language processing service to extract business intent information and the type of information to be collected from the speech-recognized text. Obtain the session process status of the voice call session, and determine the current service node from the service node mapping table based on the service intent information and the session process status; Based on the type of information to be collected and the information integrity conditions corresponding to the current business node, the speech recognition text is verified for information integrity, and an information integrity verification result is generated. When the type of information to be collected is a precise text information type, or when the information integrity verification result indicates that the speech recognition text has failed the information integrity verification, a task identifier is generated; Construct a text input task and write the current business node, the type of information to be collected, the call session identifier, and the task identifier into the text input task.

6. The method for collaborative voice and text interaction during a call as described in claim 1, characterized in that, Based on the text input task, the system receives text information to be verified through the visual interactive interface, generates a digital reference identifier bound to the text information to be verified, writes the digital reference identifier, the text information to be verified, and the call session identifier into a text mapping cache to generate a text mapping record, generates a digital input message based on the digital reference identifier, and sends the digital input message to the session control node, including: Read the task identifier, current business node, information type to be collected, and call session identifier from the text input task, and read the submission session identifier from the text interaction session. Generate a text submission control based on the task identifier, the current business node, and the information type to be collected, and load the text submission control into the visual interactive interface. The system receives the text information to be verified, the return task identifier, and the return session identifier submitted by the terminal device through the text submission control in the visual interactive interface, and collects the submission time marker corresponding to the text information to be verified. The consistency of the return task identifier and the task identifier is verified, the consistency of the return session identifier and the submission session identifier is verified, and the collection item is verified according to the current business node and the type of information to be collected, and a submission verification result is generated. When the submission verification result is deemed successful, a digital reference identifier bound to the text information to be verified is generated based on the task identifier, the call session identifier, and the submission timestamp. Write the digital reference identifier, the text information to be verified, the call session identifier, the task identifier, the current business node, and the information type to be collected into the text mapping cache to generate a text mapping record; A numeric input message conforming to the telephone keypad input format is generated based on the numeric reference identifier. The numeric input message is associated with the call session identifier and sent to the session control node through the telephone input interface.

7. The method for collaborative voice and text interaction during a call as described in claim 1, characterized in that, After the numeric input message arrives at the session control node, the numeric reference identifier is obtained based on the numeric input message. The text mapping record is read from the text mapping buffer according to the call session identifier and the numeric reference identifier. The text information to be verified is extracted from the text mapping record. An information verification result is generated based on the current service node, the speech-recognized text, and the text information to be verified, including: After the digital input message arrives at the session control node, the digital payload field is extracted from the digital input message, and the digital reference identifier is obtained based on the digital payload field; Read the call session identifier corresponding to the numeric input message from the session context of the session control node, and generate a cache read index based on the call session identifier and the numeric reference identifier; Based on the cache read index, read the text mapping record from the text mapping cache area, and read the record session identifier, record reference identifier, record business node, record call status and cached text information from the text mapping record; The consistency verification results are generated by performing consistency verification between the record session identifier and the call session identifier, and by performing consistency verification between the record reference identifier and the digital reference identifier, and by performing consistency verification between the record service node and the current service node, and by performing consistency verification between the record service node and the current service node. Finally, a record verification result is generated based on the session consistency verification result, the reference consistency verification result, the service node consistency verification result, and the record call status. When the record verification result is passed, the cached text information is extracted from the text mapping record as the text information to be verified, and the record call status in the text mapping record is updated to the called status. Based on the current business node read information verification conditions, the speech recognition text and the text information to be verified are subjected to text consistency verification and field integrity verification according to the information verification conditions, and information verification results are generated.

8. A device for collaborative voice and text interaction during a call, characterized in that, The voice and text collaborative interaction device during the call includes: The call session establishment module is used to receive a voice call request sent by a terminal device, make the voice call request reach the session control node, extract terminal device information and called number from the voice call request, establish a voice call session between the terminal device and the voice interaction node based on the called number, and generate a call session identifier corresponding to the voice call session. The text permission verification module is used to verify text interaction permissions based on the call session identifier and the terminal device information after the voice call session is established, and to establish a text interaction session bound to the voice call session when the text interaction permission verification is passed. The visual interface push module is used to send an interface creation request to the terminal device based on the text interaction session, so that the terminal device can display a visual interactive interface while maintaining the voice call session. The speech semantic recognition module is used to generate speech recognition text based on the voice call session, and determine the current business node and text input task based on the speech recognition text; The text mapping forwarding module is used to receive text information to be verified through the visual interactive interface based on the text input task, generate a digital reference identifier bound to the text information to be verified, write the digital reference identifier, the text information to be verified and the call session identifier into the text mapping cache to generate a text mapping record, generate a digital input message based on the digital reference identifier, and send the digital input message to the session control node. The text information verification module is used to obtain the digital reference identifier based on the digital input message after the digital input message arrives at the session control node, read the text mapping record from the text mapping cache according to the call session identifier and the digital reference identifier, extract the text information to be verified from the text mapping record, and generate an information verification result based on the current service node, the speech recognition text and the text information to be verified.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a voice and text collaborative interaction program stored in the memory and executable on the processor. When the voice and text collaborative interaction program is executed by the processor, it implements the steps of the voice and text collaborative interaction method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a voice and text collaborative interaction program during a call, which, when executed by a processor, implements the steps of the voice and text collaborative interaction method during a call as described in any one of claims 1-7.

Images